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What GAO Found 

FEMA has taken a number of steps to enhance the accuracy of flood maps, 
but challenges related to implementing standards to ensure map accuracy 
remain. Steps FEMA has taken include adopting a risk-based method to 
prioritize mapping projects, implementing mapping standards and guidance, 
establishing risk-based standards for topographic detail to ensure that the 
highest risk areas have the most accurate topographic data, and implementing 
quality control processes for ensuring engineering data is collected and used 
in accordance with standards. However, FEMA's mapping standards could be 
improved. For example, FEMA has standards for determining the extent to 
which new and updated flood mapping data are sufficiently current to 
promote map accuracy, yet FEMA has not developed uniform guidance for the 
validation of existing mapping data. Doing so could help FEMA both track and 
report the accuracy of maps at the national and regional levels and better 
assess mapping data needs. FEMA's quality control process for ensuring the 
accuracy of flood maps could also be improved. Audits of FEMA's mapping 
contractors' efforts have been conducted since 2006 by an independent 
verification contractor; however, FEMA officials said they planned to transfer 
responsibility for the verification audits, part of its independent verification 
and validation process, to its program management contractor by the end of 
this year, who will then monitor FEMA's mapping contractors. The transfer of 
these responsibilities creates a potential conflict of interest because the 
program management contractor is to monitor the results of its program 
management efforts. According to industry best practices, verification and 
validation efforts should be independent and reported directly to senior 
management to provide added assurance that reported results on the project's 
status are unbiased. The performance of the verification and validation 
function by an entity that is technically, managerially, and financially 
independent of the organization in charge of what it is assessing could better 
position FEMA to help ensure the independence of the verification and 
validation function, both in appearance and in fact. 

FEMA has taken a variety of steps to conduct outreach to state and local 
officials, including developing a national outreach strategy, but could 
enhance its efforts to improve public awareness and promote map 
acceptance. For example, FEMA has not developed performance goals or 
measures, or identified the resources needed for its flood mapping 
outreach efforts, which could help FEMA better determine whether its 
outreach efforts are achieving their intended results. In addition, FEMA 
could better quantify, allocate, and leverage resources needed to support 
national outreach efforts. For example, by tracking spending and using 
risk in its decisions for allocating outreach resources, FEMA could better 
allocate resources for flood mapping outreach efforts. In addition, FEMA 
could enhance its outreach efforts by leveraging existing flood insurance 
marketing resources and expertise during the mapping process to increase 
public acceptance of flood maps. 
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United States Government Accountability Office 
Washington, DC 20548 



December 2, 2010 

The Honorable Sherrod Brown 
Chairman 

Subcommittee on Economic Policy 

Committee on Banking, Housing and Urban Affairs 

United States Senate 

The Honorable Jeff Bingaman 
United States Senate 

The Honorable Charles E. Schumer 
United States Senate 

Unprecedented flooding in 2009 and 2010 in Atlanta, Georgia; Nashville, 
Tennessee; Oklahoma City, Oklahoma; and in 24 counties across Arkansas 
caused millions in property damages and heightened the nation's 
awareness of the importance of flood insurance. 1 The Federal Emergency 
Management Agency (FEMA), a component of the Department of 
Homeland Security (DHS), is responsible for mapping flood-prone areas 
across the country. Based upon the estimated flood risk reflected in these 
maps, FEMA makes flood insurance available to property owners in more 
than 20,000 communities that participate in the National Flood Insurance 
Program (NFIP). We designated the program as a "high-risk" area in March 
2006 because it likely will not generate sufficient revenues to repay the 
billions it borrowed from the Treasury to cover flood claims from the 2005 
hurricanes. 2 From 2003 through 2008, FEMA was appropriated $1.2 billion 
for a comprehensive effort to update the nation's inventory of flood 
insurance maps — known as the Map Modernization initiative. In fiscal 
years 2009 and 2010, Congress appropriated a total of $440 million for 
FEMA to continue its flood mapping efforts. 3 Federal law requires FEMA 



1 According to FEMA, 20 to 25 percent of flood claims are to communities and properties 
outside of a "Special Flood Hazard Area" (SFHA), which are those areas that have an 
estimated 1 percent annual chance of flooding. 

2 GAO's High-Risk Series identifies federal programs and operations that, in some cases, 
are high risk due to their greater vulnerabilities to fraud, waste, abuse, and 
mismanagement. GAO, High-Risk Series: An Update, GAO-07-310 (Washington, D.C.: 
January 2007). 

3 Pub. L. No. 111-83, 123 Stat. 2142, 2163 (2009); Pub. L. No. 110-329, 122 Stat. 3574, 3675 



(2008). 



Page 1 



GAO-11-17 FEMA Flood Maps 



to assess the need to revise and update the nation's flood maps at least 
every 5 years; 4 in response, FEMA reviews 20 percent of flood maps on an 
annual basis. 5 

As FEMA concluded in an August 2010 report to Congress, inaccurate 
maps create substantial difficulties by undermining confidence in the 
NFIP, leaving some individuals and organizations unaware of their risks, 
and imposing unnecessary costs on others whose risk is overstated. 6 
FEMA established its 5-year Risk Mapping Assessment and Planning (Risk 
MAP) program in 2009 to, among other things, improve the quality of flood 
data used for mapping and enhance public acceptance of flood maps. 

Ensuring the accuracy — and promoting public acceptance of — flood maps 
are ongoing challenges that FEMA faces in implementing its national flood 
mapping program, as evidenced by past reviews and recommendations 
made by us and others in assessing FEMA's efforts. For example, in our 
2004 report on FEMA's mapping program, we assessed FEMA's plans to 
match the accuracy of flood data with communities' relative flood risk and 
reported that FEMA had not yet established data standards that describe 
the appropriate level of detail, accuracy, and analysis required to develop 
digital maps based on risk level. 7 We recommended that FEMA develop 
and implement flood-mapping data standards for data collection and 
analysis for communities of similar risk. FEMA agreed and, in response, 
established a risk-based standard for the accuracy of mapping floodplain 
boundaries in 2005. In our report, we also reviewed FEMA's partnerships 
with states and local entities that conduct mapping activities and reported 
that FEMA had not yet developed a clear strategy for partnering with 
communities with few resources and little or no experience in flood 
mapping. We recommended that FEMA develop and implement strategies 
for partnering with state and local stakeholders and establish useful 



4 42 U.S.C. § 4101(e). 

° For the purpose of this report, we will use the terms flood maps to describe FEMA's flood 
insurance map products such as Flood Insurance Rate Maps (FIRM), Digital Flood 
Insurance Rate Maps (DFIRM), accompanying Flood Insurance Study and other supporting 
technical data. 

6 Federal Emergency Management Agency, Risk Mapping, Assessment and Planning (Risk 
MAP): National Digital Elevation Acquisition and Utilization Plan for Floodplain Mapping 
(Aug. 9, 2010). 

7 GAO, Flood Map Modernization: Program Strategy Shows Promise, but Challenges 
Remain, GAO-04-417 (Washington, D.C.: Mar. 31, 2004.). 
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performance measures to assess its progress in increasing stakeholders' 
awareness and use of new maps. FEMA agreed and, in 2006, cited a 
number of activities the agency had taken to increase the effectiveness of 
its mapping partnerships, including the development and implementation 
of a national outreach strategy and the creation of an outreach consortium 
to share lessons learned. Recommendations to improve flood-mapping 
data quality and community outreach have also been made by the 
Technical Mapping Advisory Council, 8 and by the Department of 
Homeland Security's Office of Inspector General. 9 

You requested that we review FEMA's flood mapping program. In 
response, this report addresses the following objectives: 

• To what extent has FEMA taken actions to enhance the accuracy of 
flood maps, and what challenges, if any, does FEMA face? 

• To what extent has FEMA taken actions to help promote community 
acceptance of flood maps, and what challenges, if any, does FEMA 
face? 

To address our first objective, we assessed FEMA's standards and 
guidance against criteria in recent reports by the National Academies of 
Sciences and the National Research Council. 10 We discussed the reports' 
methodologies with the authors and with relevant FEMA officials, and 
analyzed reviews and critiques of the Academies' reports to determine that 
they were appropriate for our purposes. We analyzed information on 
FEMA's policies and plans for flood map modernization, data from 
FEMA's Mapping Information Platform (MIP) for the period of October 
2005 through 2009, and systems for documenting compliance with FEMA's 



The Technical Mapping Advisory Council was established by the National Flood 
Insurance Reform Act of 1994 to provide recommendations to FEMA on how to improve 
the accuracy, quality, distribution, dissemination, and ease of use of Flood Insurance Rate 
Maps, among other things. Pub. L. No. 103-325, §576, 108 Stat. 2255, 2280 (1994). The 
Council was created in November 1995 and it continued through November 2000. The 
Council submitted recommendations to the Director of FEMA in each of its Annual 
Reports. 

9 Department of Homeland Security Office of the Inspector General, Challenges in FEMA's 
Flood Map Modernization Program, OIG-05-44 (Washington, D.C.: September 2005.). 

10 National Research Council (U.S.), and United States. 2009 Mapping the Zone: Improving 
Flood Map Accuracy. Washington, D.C.: National Academies Press. 
http://www.nap.edu/catalog.php?record_id= 12573 May 2009; National Research Council 
(U.S.). 2007 '. Elevation Data for Floodplain Mapping. Washington, D.C.: National 
Academies Press. http://books.nap.edu7catalog.php?record_id= 11829 August 2007. 
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data quality standards. 11 To assess FEMA's internal controls and the 
reliability of computer-processed flood map data, we examined FEMA 
databases, including the MIP, which was designed to monitor the mapping 
process and the completion of FEMA's quality assurance/quality control 
(QA/QC) process. We tested the controls on the QA/QC process by 
extracting and reviewing data on all projects initiated and completed from 
fiscal years 2006 through 2009. We also analyzed FEMA's Floodplain 
Boundary Standard (FBS) and New, Validated, or Updated Engineering 
(NVUE) verification systems that were designed to track implementation 
of data accuracy requirements. We tested the controls on the FBS and 
NVUE compliance process by extracting and reviewing data on all projects 
initiated and completed from fiscal year 2006 (when the FBS was 
established) through 2009 and compared them against criteria in 
Standards for Internal Control in the Federal Government. 12 To assess 
the reliability of these databases, we compared data to FEMA's 
management reports, interviewed FEMA's three mapping contractors, and 
analyzed the original data. We determined that the FBS and NVUE 
compliance data were sufficiently reliable for our purposes. We also 
discussed FEMA's mapping process and standards with agency officials, as 
well as other federal stakeholders in geographic data collection and 
mapping, including officials at the U.S. Geological Survey, the National 
Oceanic and Atmospheric Administration, the U.S. Army Corps of 
Engineers, and subject-matter experts on flood hazards and floodplain 
management from national organizations, including the Association of 
State Floodplain Managers and the National Association of Flood & 
Stormwater Management Agencies, which are stakeholders to FEMA's 
mapping initiatives. We focused our review on FEMA's standards and 
processes related to flood hazard mapping for rivers and streams 
(commonly know as "riverine" flooding 13 ), which account for about 95 
percent of FEMA's flood maps, according to FEMA. As a result, we limited 
our scope to exclude those standards and processes related to flood 
hazard mapping for coastal areas and the levee certification. In addition, 



FEMA created the Mapping Information Platform in 2004, to enable the management, 
production, and sharing of flood hazard data and maps and related information in a digital 
environment. In March 2006, FEMA developed a Mid-Course Adjustment to its Map 
Modernization Initiative. 

12 GAO, Standards for Internal Control in the Federal Government, GAO/AIMD-00-21.3.1 
(Washington, D.C.: November 1999). 

13 Riverine flooding is flooding related to or caused by a river, stream, or tributary 
overflowing its banks due to excessive rainfall, snowmelt, or ice. 
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FEMA has processes to modify and update flood map information during 
the time that a community's maps are in effect, called a Letter of Map 
Change, which is also outside the scope of our work. 

To address our second objective, we analyzed information on FEMA's 
policies, requirements for community outreach, and data from FEMA's 
information management systems (discussed above) for documenting 
compliance with statutory and regulatory requirements for documenting 
coordination with state and local officials involved in mapping projects. To 
determine FEMA's compliance with documentation requirements, we 
examined FEMA's Flood Elevation Determination Dockets (FEDD) files 
that are established for each mapping project. We reviewed FEDD files 
from a probability sample of 88 counties from a population of 431 counties 
that had completed studies from fiscal year 2006 through 2009 that 
resulted in a change in base flood elevation. 14 From this sample, we 
reviewed mapping partners' compliance with six documentation 
requirements. 15 In addition, we analyzed the goals and performance 
measures of FEMA's outreach strategy for Map Modernization, and its 
Risk MAP national outreach strategy against prior GAO work reviewing 
federal agencies' practices for development of national strategies, 16 as well 
as FEMA's budget and staff allocations related to outreach. 

To supplement our analyses of FEMA's flood mapping internal controls 
and program management activities related to data accuracy and 
community outreach, we selected four flood map modernization projects 
in Arizona, California, Florida, and North Carolina to visit. We selected 
these locations based on our 2004 review to highlight specific challenges 
associated with the mapping process, such as inclusion of levees, and the 
impact of varying degrees of community involvement and outreach in the 5 
years since our review. The results from these locations cannot be 
generalized to all flood map modernization projects, but enabled us to 
describe challenges FEMA faces in conducting its national flood mapping 



The maximum margin of error for estimates of percentages from our sample of flood 
mapping projects is plus or minus 9 percentage points at the 95 percent level of statistical 
confidence. 

15 FEMA's requirement to maintain a flood elevation determination docket (FEDD) is found 
at 44 C.F.R. § 67.3. 

16 See GAO Combating Terrorism: Evaluation of Selected Characteristics in National 
Strategies Related to Terrorism, GAO-04-408T (Washington, D.C., Feb. 3, 2004), and GAO 
Financial Literacy and Education Commission: Further Progress Needed to Ensure an 
Effective National Strategy, GAO-07-100 (Washington, D.C., Dec. 4, 2006). 
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activities by talking with relevant state and local officials. An expanded 
discussion of our scope and methodology is described in appendix I. 

We conducted this performance audit from August 2009 through 
December 2010, in accordance with generally accepted government 
auditing standards. Those standards require that we plan and perform the 
audit to obtain sufficient, appropriate evidence to provide a reasonable 
basis for our findings and conclusions based on our audit objectives. We 
believe that the evidence obtained provides a reasonable basis for our 
findings and conclusions based on our audit objectives. 



Background. FEMA is the primary federal agency responsible for assisting state and 

" local governments, private entities, and individuals to prepare for, 

mitigate, respond to, and recover from natural disasters, including floods. 
Floods are the most frequent natural disasters in the United States, 
causing billions of dollars of damage annually. To address the increasing 
amount of flood damage, the lack of readily available insurance for 
property owners, and the cost to the taxpayer for flood-related disaster 
relief, Congress passed the National Flood Insurance Act of 1968, which 
created the NFIP. 17 Since its inception, the NFIP has served as a key 
component of FEMA's efforts to minimize or mitigate the damage and 
financial impact of floods on the public, as well as to limit federal 
expenditures needed after floods occur. The NFIP seeks to minimize 
flood-related property losses by making flood insurance available on 
reasonable terms and encouraging its purchase by people who need flood 
insurance protection — particularly those living in the areas at highest risk 
of flooding, known as Special Flood Hazard Areas, designating a 1 percent 
annual chance of flooding. To do so, FEMA along with its state and local 
partners, identifies and maps flood-prone areas in the more than 20,100 
communities that currently participate in the program. 18 

When the NFIP was created, the purchase of flood insurance was 
voluntary. Congress amended the original law in 1973 to require the 
purchase of flood insurance in certain circumstances. The purchase of 
flood insurance is required for structures in Special Flood Hazard Areas of 
communities participating in the program if (1) any federal loans or grants 
were used to acquire or build the structures or (2) the structures have 



17 Pub. L. No. 90-448, tit. XIII, 82 Stat. 572 (1968). 

18 Also included are Puerto Rico and the Virgin Islands. 
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outstanding mortgage loans made by lending institutions that are regulated 
by the federal government. Property owners located in the Special Flood 
Hazard Area with mortgages from federally regulated lenders are required 
to purchase and maintain flood insurance policies. 

FEMA identifies flood hazards, assesses flood risks, and provides 
appropriate flood hazard and risk information to communities nationwide. 
To identify hazards and assess risks, mapping projects are performed in 
accordance with FEMA Guidelines and Specifications. Flood maps 
provide the basis for setting insurance rates and identifying properties 
whose owners are required to purchase flood insurance. FEMA's flood 
hazard maps are also used by lending institutions to determine who is 
required to purchase flood insurance and help ensure that flood insurance 
is purchased and maintained for these properties. Local government 
planning and zoning officials, land developers, and engineers use the maps 
for developing zoning regulations and designing new buildings and 
infrastructure to be safe from flooding. FEMA has estimated that local 
governments' compliance with the program's standards for new 
construction saves over $1 billion annually in flood damage avoided. 



Stakeholders from All 
Levels of Government and 
the Private Sector 
Participate in the Mapping 
Process 



FEMA's Federal Insurance and Mitigation Administration, which manages 
the NFIP, is comprised of three divisions: Risk Analysis, Risk Reduction, 
and Risk Insurance. 19 The Risk Analysis Division is responsible for flood 
mapping activities and develops flood mapping policy and guidance. 
FEMA's 10 Regional offices manage flood map production for their 
geographic areas. FEMA headquarters and regional staff monitor and 
report flood hazard mapping progress based on program management data 
provided by flood mapping partners. 



Mapping partners can include FEMA's 3 national Production and 
Technical Services (PTS) contractors, as well as state and local 
governments or regional agencies — including those state and local 
governments that are participating in FEMA's Cooperating Technical 
Partners (CTP) program. The PTS contractors are private engineering 



FEMA's Risk Reduction Division performs floodplain management activities to reduce 
risk to life and property through the use of land use controls, building practices and other 
tools, in both pre- and post-disaster environments. FEMA's Risk Insurance Division 
provides flood insurance for property owners and encourages communities to adopt and 
enforces floodplain management regulations that mitigate the effects of flooding on new 
and improved structures. 
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firms working under contract to FEMA and are each responsible for a 
regional portfolio of flood study projects. Table 1 summarizes the roles 
and responsibilities of FEMA, the mapping partners, and other actors in 
the flood map production process. 



Table 1 : Mapping Partners Roles and Activities in the Flood Map Production Process 



Stakeholders 


Requirement or responsibility or role 


FEMA headquarters staff 


• Manage national flood-mapping program 




• Monitor local governments' adoption of maps and updates to ordinances 


FEMA regional staff 


• Oversee scoping meeting between all mapping partners 




• Manage flood-mapping process 




m I oaH rriDoHnnc fr\ nroconr r~i kg I i m inan/ mane \ft/ith Iz-m^q I nn\/ornmont r^ffi/^iQlc 
* LccHJ I I IcfcJLII ILjo LU (JlfcJofcJIlL fJltJIII 1 III Ictl y 1 1 ld[Jo Willi lUOcll yu V fcfl 1 1 1 1 ItM 1 L UIIILrldlo 




• Provide local government officials with outreach tools 




• Attpnrl nnhlir mpptinnc; 

w niLCI IvJ |JUL/IHj 1 1 ICC LI 1 i y o 


Mapping partners: 


• Assist in scoping meeting 


• Contractors 


• Collect required data or validates existing data 


• Cooperating Technical Partners 


• Assist FEMA in administering flood-mapping activities (see text above) 


(CTPs) 


• Analyze flood hazard data sources (ie. climate, stream flow, soil, land use, 


• Other federal agencies 8 


elevation, hydraulic structure) 




• Produce flood hazard estimates 




• Implement quality controls 




• Create preliminary maps 




• Attend public meetings 




• Resolve appeals and /or protests to preliminary maps 




• Create final maps 


State & local officials" 


• Participate in scoping meeting 




• Identify data assets and needs during scoping meetings 




• Provide feedback on preliminary maps 




• May conduct outreach to individuals in the community 




• Collect appeals and/or protests to the preliminary maps from individuals and 




forwards to FEMA 




• Update local governments' floodplain ordinances 


Community/general public (e.g., property 


• Attend public meetings 


owners, businesses, local real estate 


• Provide feedback on preliminary maps 


industry, etc.) 


• May file challenges— appeals and/or protests — to preliminary maps 



Source: GAO analysis of FEMA data. 

"Other Federal agencies who work under interagency agreements with FEMA could include, for 
example, the U.S. Army Corps of Engineers, Natural Resources Conservation Service, U.S. 
Geological Survey, National Oceanic and Atmospheric Administration, and the Tennessee Valley 
Authority. 

"State and local officials can also be Cooperating Technical Partners. 
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FEMA relies on local governments to provide it with notification of 
changing flood hazard information and to work with FEMA to collect the 
information needed to reflect the updated flood hazards on the flood 
maps. Changes to communities such as new development can affect 
floodplain boundaries, as shown in figure 1 below. Thus, as we noted in 
our 2004 report, the ultimate success of FEMA's flood mapping program 
depends on the level of community investment and involvement in the 
process. 20 



Figure 1 : Effects of Development on a Riverine Floodplain 




Sources: GAO analysis of FEMA data; and Art Explosion clipart. 



A community's flood hazard maps can be updated in response to a FEMA- 
initiated study or revised study of flood hazards and subsequent revision 
of NFIP flood maps or through a community-initiated revision. Each year, 
FEMA revises existing maps in communities across the nation. Because of 
funding constraints, FEMA can study or revise maps for only a limited 



GAO-04-417. 
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number of communities each year. As a result, FEMA prioritizes new and 
revised study needs based on a cost-benefit approach whereby the highest 
priority is given to studies where development is greatest and where the 
maps are most outdated. 



Overview of Flood 
Mapping Production 
Process 

Topographic Accuracy 

Topographic accuracy is a function of detail 
and age. Detail is important because 
detailed topography has significantly fewer 
errors than less detailed alternatives and 
better accounts for hydraulic 
structures — structures that affect water 
flow — such as buildings, dykes, river banks, 
and roads. Age is important because 
topography can change over time due to 
development and ecological factors such as 
erosion. The topographic data used in 
mapping studies can have significant 
variances in age and detail, and thus, 
accuracy. 

Source: GAO analysis. 

Base flood elevation (BFE) 

The computed elevation of a flood having a 1 
percent chance of being equaled or 
exceeded in a given year is the base flood 
elevation. It accounts for the volume and 
velocity of water moving through the 
watershed and reflects the cumulative effects 
of topography, soils, vegetation, surface 
permeability, and other factors. The BFE is 
the regulatory standard for the elevation or 
flood proofing of structures, and the 
relationship between the BFE and the 
elevation of a structure also determines the 
flood insurance premium. In general, the 
higher the first floor elevation, the lower the 
insurance premium. Consequently, the 
accuracy of BFEs on the flood maps is 
important for both regulating and insuring 
properties commensurate with the risk of 
flooding. 

Source: National Academies of Science 



Flood mapping is a complex and technical endeavor. In order to create a 
map, engineers must conduct field surveys to assess the area to be studied 
and then develop data on the elevation of the terrain — called topographic 
data. 21 Engineers develop flood hazard data that estimates the risk of 
flooding by performing analyses on the hydrologic conditions that affect 
the amount of water that flows downstream during a flood (for example, 
soil and vegetation absorb rain and reduce runoff while pavement and 
other impermeable manmade surfaces increase the flow of runoff) and the 
hydraulic conditions that affect the height of floodwaters in streams or 
waterways (for example, bridges may create narrower channels that raise 
the water level as it passes under the bridge). 

The results of the analyses of these different types of topographic and 
flood hazard engineering data are then combined and integrated into 
digital maps that describe how far (the floodplain boundary) and how high 
floodwaters will reach (the Base Flood Elevations, or BFEs) — as shown 
in figure 2 below. Each step in the process contributes to the ultimate 
accuracy of the final map but also requires judgment and involves 
uncertainty. Without a long and well-documented record of flooding in a 
floodplain, the precision of flood hazard information is difficult to 
determine. Because weather predictions and land use are difficult to 
predict, the correctness of the flood maps cannot be determined with 
certainty. Instead, the maps must be evaluated based on a relative 
correctness or general reliability of the flood maps and flood insurance 



According to FEMA officials, the term "field survey" within FEMA usually means actual 
survey measurements made by a crew on the ground, and the term "field reconnaissance" is 
usually used within FEMA to distinguish site visits to get an overall understanding of the 
area to be studied and collect information other than survey measurements. In addition, 
topographic data is mostly produced by aerial surveys, although certain key features are 
often measured by field survey because the aerial survey technology may not produce 
accurate results in some situations. 
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study. 22 Some of the factors that impact reliability of the study are the type 
of topographic data used, the hydrologic and hydraulic models used and 
the assumptions computed, and the final mapping techniques by the 
mapping partners. 



Figure 2: Riverine Floodplain Boundary and the Base Flood Elevation 



Special Flood Hazard Area 




Sources: GAO analysis of FEMA data; and Art Explosion clipart. 

Through various stages of the mapping production process, FEMA, in 
consultation with mapping partners and localities, determines the flood 
map study's level of accuracy and precision. In making this decision, 
required costs and resources, budget priorities, and communities' flood 
hazard identification needs are considered. An overview of the mapping 
production process is provided in figure 3 below. This determination 
affects the study's cost and the resulting flood map's accuracy. Detailed 
flood studies incorporate greater amounts of data or more precise data 
into a map to provide greater granularity of information, for example, by 
determining BFEs within a Special Flood Hazard Area, to reduce 
uncertainty. In contrast, approximate flood studies generally require less 
precision in flood hazard data. For example, they are used for areas that 
are less subject to development and do not require the establishment of a 
regulatory base flood elevation, although base flood elevations may be 
identified on the flood map based on an agreement between FEMA, its 



22 A Flood Insurance Study (FIS) is a book that contains information regarding flooding in a 
community and is developed in conjunction with the flood insurance rate map. The FIS, 
also known as a flood elevation study, frequently contains a narrative of the flood history 
of a community and discusses the engineering methods used to develop the maps. The 
study also contains flood profiles for studied flooding sources and can be used to 
determine Base Flood Elevations for some areas. 
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mapping partners, and state and local governments. Even when 
floodplains are mapped with high accuracy, land development and natural 
changes to the landscape or hydrologic systems create the need for 
continuous map maintenance and updates. 



Figure 3: Map Production Process Overview 




Sources: GAO analysis of FEMA data. 



Key: 

A - Local needs identified - FEMA, mapping partner, and local government officials meet to discuss 
a plan and schedule a Scoping Meeting(s) for flood mapping project, including data needs. 

B - Data Development period - Mapping partner collects data or validates existing data and uses a 
model to create preliminary maps. All maps inevitably contain some uncertainty because of 
technological limitations and budgetary constraints. Preliminary maps are presented to local 
government officials to identify issues. FEMA publishes notifications of new flood maps in the Federal 
Register and twice in the local newspaper. Local officials may decide to hold public meetings. 

C - Regulatory Adoption Period 
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Appeal Period - 90-day period for local government officials or individuals to challenge the 
preliminary maps as scientifically or technically inaccurate because they include, for 
example, inaccurate flood levels, flood boundaries not matching elevation of terrain, 
incorrect street names, city limits, etc. - After all appeals are resolved, FEMA sends a 
Letter of Final Determination (LFD) to the community CEO to say the maps are now "final." 

Adoption Period - After maps are final the local government must update its ordinances 
within 6 months. If the local government does not update its ordinances, it could be 
suspended from the National Flood Insurance Program. 

D - Map Maintenance - After maps are final, the local government or individuals can file a "Letter" to 
modify or update an individual property or parcels of land within the flood maps for reasons such as 
new development , mitigation efforts, or the limitation of map scale and the collection of more 
accurate ground elevations data on a given property. In addition, map maintenance also includes re- 
analysis and revision of maps - not by letter, but by republishing the entire map - based either on 
information provided by local government or by FEMA's identification of an update need. 

The National Flood Insurance Act of 1968, as amended, and federal 
regulations require that FEMA communicate potential changes in flood 
risk to the public when it decides to initiate a flood mapping study and 
when it is ready to release preliminary maps. At the beginning of the 
mapping process, FEMA is required to notify local governments. 23 When 
FEMA is ready to release preliminary maps, the agency must publish the 
proposed base flood elevations in the Federal Register for public comment 
and notify the local government of the results of the study. 24 When the final 
map is approved, FEMA publishes another Federal Register notice. 25 
FEMA is required to maintain documentation of selected elements of its 
public notification efforts. 26 Outside of these statutory and regulatory 
requirements, FEMA has historically focused its outreach efforts on local 



FEMA is required to contact community stakeholders, such as the state coordinating 
agency and other appropriate community officials, to discuss the intent and nature of the 
proposed flood map study. 44 C.F.R. § 66.5. 

24 FEMA is required to publish the proposed flood elevations in a prominent local 
newspaper at least twice during the 10-day period following the notification of the 
community chief executive officer. Property owners have 90 days from the second 
newspaper publication to appeal the proposed flood elevations. 44 C.F.R. §§ 67.4, 67.5. 

25 Final flood elevations must be published in the Federal Register and copies sent to the 
community chief executive officer, all individual appellants, and the state-coordinating 
agency. 44 C.F.R. § 67.11. 

26 44 C.F.R. §§66.3, 67.3. 
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government officials and has relied on local officials to inform the 
community at large (i.e., the public) of flood mapping efforts. 27 



Map Modernization to Risk Traditionally, flood maps were created and stored in paper format. In the 
MAP early 1990s, however, some of the data and information FEMA collected to 

develop flood maps started becoming available in digital format. In 1997, 
FEMA developed its initial flood Map Modernization plan that outlined the 
steps necessary to update the nation's flood maps to digital format and 
streamline FEMA's operations in raising public awareness of the 
importance of the maps and responding to requests to revise them. 
FEMA's initial flood Map Modernization plan was to fully digitize all flood 
maps in the nation, first, by identifying those maps that required 
engineering updates and converting them to a digital format. FEMA's 
initial goal was to convert approximately 80 percent of existing paper 
maps to a digital format, update 20 percent of the existing maps with new 
flood risk information while converting them to digital format, and add 
13,700 completely new maps (also in digital format) to cover previously 
unmapped communities. Then, a planned maintenance phase would follow 
the Map Modernization initiative, whereby these maps would be updated 
with new engineering data. 

In March 2006, FEMA performed a mid-program evaluation that 
considered input from our prior work, as well as the Congress, the 
Department of Homeland Security's Inspector General, and other 
stakeholders. As a result, FEMA instituted a mid-course adjustment of the 
Map Modernization's program goals and objectives. FEMA's modified 
objectives for the initiative were to (1) produce new digital products; (2) 
provide new, updated, or validated engineering analyses; and (3) integrate 
a new Floodplain Boundary Standard into the digital maps. As part of this 
mid-course adjustment, FEMA ranked all 3,146 counties in the United 
States in terms of flood risk from highest to lowest based on a number of 
factors, including, among other things, population, growth trends, housing 
units, flood insurance policies and claims, repetitive loss properties, and 
flood disasters. On the basis of this ranking, FEMA established its mapping 
priorities that the agency used to schedule mapping projects during the 



Federal law provides that FEMA must encourage local officials to disseminate 
information concerning a flood mapping study widely within the community, so that 
interested persons will have an opportunity to bring all relevant facts and technical data 
concerning the local flood hazard to the attention of the agency during the course of the 
study. 42 U.S.C. § 4107. 
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course of its Map Modernization initiative, which FEMA detailed in their 
Multi-Year Flood Hazard Identification Plans. From fiscal years 2003 
through 2008, FEMA spent $1.2 billion for flood map modernization. FEMA 
initiated the final year of production under Map Modernization in 2008. 28 

In fiscal year 2009, FEMA began a 5-year effort— Risk MAP — with $300 
million in funding from the National Flood Insurance Fund and 
congressional appropriations for flood hazard mapping. According to 
FEMA, the vision for Risk MAP is to deliver quality data that increases 
public awareness and leads to action that reduces risk to life and property. 
According to FEMA's Risk MAP, Quality Assurance Management Plan, 
quality data is defined as accurate, credible, timely, and efficiently 
delivered. 



FEMA Has Taken 
Steps to Enhance 
Flood Map Accuracy, 
but Faces Challenges 
in Implementing 
Standards and Its 
Quality Assurance 
Process for Program 
Management 



FEMA has implemented and tracks compliance with three standards for 
ensuring the quality of data used in developing flood maps: FEMA's 
Guidelines and Specifications; the Floodplain Boundary Standard (FBS), 
and 2 of the 3 elements of the New, Validated and Updated Engineering 
(NVUE) data standard. 



FEMA Has Developed 
Standards and a Quality 
Assurance Process to 
Enhance Map Accuracy 



According to FEMA officials, production of these 2008 projects under Map Modernization 
continued through 2010, and some of these projects were still in progress at the time of our 
review. 
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• Guidelines and Specifications for Flood Hazard Mapping Partners — 
FEMA established the Guidelines and Specifications to define 
technical requirements, product specifications for Flood Hazard Maps 
and related NFIP products, and associated coordination and 
documentation activities. In addition, FEMA periodically amends the 
Guidelines and Specifications, through Procedural Memoranda. For 
example, in September 2010, FEMA revised Guidelines and 
Specifications for acquiring elevation to include risk-based standards 
for gathering and using topographic data. 

• Floodplain Boundary Standard — In response to stakeholders concern 
about the quality of flood data used to develop new flood maps during 
the Flood Map Modernization program, FEMA issued the FBS in 
October 2007, in part, to help ensure that flood maps are tied to a 
topographic source. The purpose of the FBS is to ensure the locations 
of the predicted horizontal (floodplain boundary) and vertical (base 
flood elevation) lines drawn on flood maps are comparable to the 
topographic data that has been selected for the study area. For 
example, maps showing water running uphill could occur if 
inaccuracies existed when calculating the base flood elevation against 
the topographic data, according to mapping contractors. The FBS 
reduces the chance of such errors taking place, which enhances the 
public credibility of flood maps, according to mapping contractors. All 
studies contracted since 2006 must comply with the FBS. In FEMA's 
2006 Mid-Course Adjustment to the Map Modernization program, the 
agency set a goal that 75 percent of stream miles reflected on FEMA's 
issued maps were to be compliant with FBS by the end of Map 
Modernization. 29 FEMAs last quarterly report of fiscal year 2009 
indicated that FEMA had met this goal; according to the report, flood 
maps have reached approximately 89 percent stream mile compliance 
with the FBS nationally. 

• New, Validated, or Updated Engineering (NVUE) standard — FEMA 
also developed a standard called the New, Validated, or Updated 
Engineering (NVUE) standard to provide a basis for assessing the 
engineering analysis used to develop flood elevations. FEMA developed 
the standard to help mapping partners determine where new study data 
should be collected, where updates to existing flood hazard data 



The total number of stream miles includes approximately 4.2 million miles of channels 
(waterways and rivers) and 600,000 miles of coastline shorelines (open ocean, lakes, and 
ponds) in the United States, according to the National Research Council, Committee on 
Floodplain Mapping Technologies. 
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should be performed, and whether previously developed flood study 
data could still be considered valid. FEMA issued draft guidance for 
validating existing data in April 2007. FEMA intends to use the NVUE 
data quality standard in implementing its Coordinated Needs 
Management System. According to officials, the Coordinated Needs 
Management System is an assessment tool to determine mapping needs 
and a means for making funding allocation decisions. FEMA officials 
stated that the Coordinated Needs Management System is to provide 
FEMA with a national assessment of data needs. However, the system 
is under development and is projected to be implemented nationwide 
in 2011. According to FEMA officials, the Coordinated Needs 
Management System is to track information including the flood zone 
designation, risk assigned, study type, and the date the analysis was 
completed or validated. FEMA believes that this approach will provide 
better detail regarding the precision of a flood hazard analysis for end 
users that they can understand, and will be applied in a risk-based 
manner. 

To monitor the quality of the process used throughout development of 
flood maps in accordance with the standards in the Guidelines and 
Specifications, FEMA established the Mapping Information Platform (MIP) 
information system. The MIP provides mapping partners the tools and 
technology to create, validate, store, track and update flood data 
according to FEMA's standards using the MIP's map production processes. 
FEMA also developed quality assurance management plans and processes 
to work with local communities and flood mapping partners. FEMA's 
quality management plans identify quality assurance steps that are to 
occur during the creation, review, and editing of flood hazard study. On 
December 1, 2008, FEMA issued revised guidance for seven quality control 
reviews to be performed during the flood map production process. Figure 
4 below provides an overview of FEMA's three data quality standards and 
the quality assurance process in the context of the steps in the mapping 
process. 
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Figure 4: FEMA's Map Process Steps and Map Accuracy Standards 





Sources: GAO analysis of FEMA data. 



In addition, FEMA established a quality assurance management system 
under both their Map Modernization and Risk MAP efforts to ensure that 
mapping products and processes comply with FEMA's specified 
requirements. Included in FEMA's quality assurance system are audits of 
the mapping process by Independent Verification and Validation (IV&V) 
audits. The IV&V contractor is to independently provide feedback to 
FEMA as part of the audit, such as sampling it conducted on the results of 
mapping projects performed by mapping partners. We previously reported 
that world-class, private sector corporations successfully employ best 
practices with quality assurance by using process controls to design 
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products and by controlling production processes as the production is 
occurring. 30 FEMA's quality management system and quality assurance 
process reflects the recognized best practice of reviewing the quality of 
the map product during the production process. 

FEMA Enhanced Its Guidelines FEMA has recently published a quality standard to set a minimum level of 
for Topographic Data by topographic detail for all studies in its Guidelines and Specification. 

Establishing Risk-Based Specifically, it established standards for the level of topographic detail 

Standards required to ensure that the maps of those areas at the highest risk from 

flooding have the most accurate topographic data, as suggested by the 
National Research Council and FEMA's Risk MAP strategy. In September 
2010, FEMA published Procedural Memorandum 61 to update its 
Guidelines and Specifications requiring mapping partners to align 
FEMA's topographic data specifications to levels of risk for flooding, as 
well as account for differing characteristics of elevation that can affect the 
accuracy and precision of base flood elevations. This procedural 
memorandum identifies the specifications of elevation accuracy and 
precision needed based on FEMA's previously-identified risk classes for all 
3,146 counties in the United States. As the National Academies of Sciences 
report stated, the level of detail used in a study should correspond to the 
area's risk. FEMA officials stated that they will only be starting new 
studies in areas where there are already existing updated and accurate 
topographic data or in areas that have sufficient need and risk to 
necessitate FEMA's funding the acquisition of such data. 

Prior to the issuance of Procedural Memorandum 61 in September 2010, 
FEMA delineated floodplains using the "best available" existing 
topographic data for the area being studied. In the absence of data 
provided by the mapping stakeholder or newly developed for a flood 
mapping project, a primary source for topographic data was the National 
Elevation Dataset maintained by the U.S. Geological Survey (USGS), 31 
which is over 35 years old on average. FEMA's existing standards for new 
topographic data required data that is about 10 times more accurate than 
USGS topographic data and required topographic data acquired or 



G AO, Best Practices: Commercial Quality Assurance Practices Offer Improvements for 
DOD, NSIAD-96-162 (Washington, D.C.: Aug. 26, 1996). 

31 The National Elevation Dataset is the primary elevation data product of the U.S. 
Geological Service that contains the best available elevation data of the United States. 
According to the Service, the data set is updated on a two month cycle to integrate any 
newly available, improved elevation source data that are processed to a common 
coordinate system and unit of vertical measure. 
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reviewed within the last 7 years to account for changes such as human 
development. FEMA plans to work with local officials to determine 
whether the existing data held by the locality or another source (such as 
the USGS), meets new the new standards, or to develop new data through 
means such as Light Detection and Ranging (LIDAR). As illustrated in 
figure 5 below and as we reported in 2004, FEMA has promoted the use of 
LIDAR remote sensing technologies to generate highly accurate, digital 
elevation data. The illustration shows an airplane equipped with laser- 
pulsing sensors using LIDAR to gather digital elevation data to measure 
the contours and crevices that determine where floodwaters collect. 
Elevation data are a key component needed to determine flood risk and 
identify floodplain boundaries. According to FEMA, for very flat areas 
where small changes in elevation can have a large impact on where flood 
plain boundaries are drawn, LIDAR can provide the level of detail needed 
to accurately delineate these boundaries. Communities can also use 
detailed, digital elevation data for planning and land development 
purposes. FEMA expects that LIDAR will be the primary technology used 
to acquire new digital elevation data for Risk MAP. 
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Figure 5: Light Detection and Ranging Technology Used to Generate Digital Elevation Data 




Source: GAO analysis of FEMA data and Art Explosion clipart. 



In a study commissioned by FEMA and issued in 2009, the National 
Academies of Sciences compared the effect of using USGS data versus 
LIDAR data at three geographic locations using the same hydrologic and 
hydraulic models. For each area studied, the two different sources of 
topographic data resulted in different BFEs and floodplain boundary 
locations. The report concluded that the quality of topographic data is the 
most important factor in determining water surface elevations, base flood 
elevations, and the extent of flooding and, thus, the accuracy of flood 
maps for riverine areas, which account for approximately 95 percent of 
FEMA's flood maps. FEMA officials agreed that accurate data are essential 
and that even the best models cannot produce an accurate flood map with 
inaccurate inputs, but they said there is a point of diminishing returns 
where the cost of developing highly accurate topographic data outweighs 
its overall benefit. 
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Prior to fiscal year 2010, FEMA did not generally provide funding for 
mapping partners to acquire new topographic data in an effort to conserve 
resources and share responsibilities, according to FEMA officials. 32 
Historically, studies at all risk levels could have used the USGS National 
Elevation Dataset as the best available data, if obtaining better quality data 
was unaffordable, according to FEMA officials. Officials from the 
Association of State Floodplain Managers agreed with this 
characterization of historical mapping efforts and said that cost 
constraints limit local governments and mapping partners' ability to 
collect extensive data, a situation that has resulted, in some cases, in poor 
map quality. FEMA officials acknowledged that affordability issues have 
been the main reason high risk areas may rely on USGS data for their 
study. To address this issue, FEMA officials said they planned to provide 
$80 million in funding in fiscal years 2010 through 2013 to acquire new 
topographic data. 



FEMA's Existing Data 
Quality Standards Could 
Be Better Implemented to 
Match Mapping Data 
Precision with the Level of 
Flood Risk 



FEMA's Floodplain Boundary Standard (FBS) reporting does not reflect 
the quality of topographic data or level of study detail, and according to 
FEMA officials, 1 of the 3 elements of the standard for New, Validated, and 
Updated Engineering (NVUE) data has not been fully implemented. 



The Floodplain Boundary 
Standard Could More 
Effectively Measure Map 
Accuracy 



The accuracy of a map's floodplain boundary, in applying the FBS for an 
area, is dependent on two factors — the quality of topographic data and the 
level of detail used for a flood study. However, the method FEMA uses in 
measuring FBS compliance does not account for these factors when 
reporting FBS compliance rates for counties. Thus, two maps using 
topographical data of widely varying accuracy and currency, and based on 
studies of different levels of detail, can both be considered FBS compliant 
as long as the base flood elevation and floodplain boundaries are 
consistent with the topographic data used in each study. Consequently, 
FBS compliance rates across counties do not provide a means for FEMA 
management to compare the relative accuracy of maps. According to 



The limited circumstances under which FEMA would historically provide funding for 
topographic data are detailed in FEMA's Geospatial Data Coordination Implementation 
Guide (v2.2), (March 2008) p. 68. Generally, FEMA only provides funding when existing 
elevation data are inadequate, i.e., do not meet the minimum accuracy required in FEMA 
flood mapping standards. 
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FEMA, the FBS was created to address problems that arose when existing 
paper map floodplain boundaries were transferred to digital format; the 
purpose was to ensure that mapping partners checked or revised 
preliminary maps to confirm they were consistent with available 
topographic data, not to compare map accuracy. The FBS, which is 
designed to help ensure the accuracy of floodplain delineations, is an 
important FEMA measure of the quality of flood maps. This is because 
individuals living within the floodplain boundary are considered to be in a 
Special Flood Hazard Area and are thereby required to purchase flood 
insurance while those outside of the boundary are exempt from this 
requirement. 33 

Stakeholders involved with the four mapping projects we contacted said 
that the quality of FBS as a measure of accuracy is only as good as the 
quality of the topographic data that measures land elevation. Because 
FEMA's standard has historically been that the mapping stakeholders 
should use the best available topographic data, the quality (including 
precision) of the topographical data used to match up with the floodplain 
boundary may vary widely depending upon the quality of what is available. 

The level of detail used in a flood study determines the requirements 
necessary for a map to achieve FBS compliance. Detailed studies have 
compliance standards more stringent than approximate studies that use 
less accurate, often outdated topographic information and models, 
according to the National Academies of Sciences report. According to 
FEMA, the determining factors when deciding whether to perform detailed 
or approximate studies for communities are the level of flood risk in the 
area, the likelihood of additional development, and the cost and benefits of 
performing a detailed study versus an approximate study, as detailed 
studies are significantly more expensive than approximate studies. Though 
FEMA advises against it, some communities in the highest risk class have 
chosen to undergo approximate studies due to fiscal restraints. 

Differences in the level of detail used in studies leads to significant 
variances in how precisely base flood elevations must match the elevation 
data used to comply with the FBS. The BFEs in detailed studies are 
required to be much more precise than those in approximate studies, but 
each are considered to be equally compliant as long as they meet the 



Flood insurance purchase is mandatory for all federally backed mortgages for properties 
in special flood hazard areas. 
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standards for their respective level of detail. For example, the BFE in 
detailed studies are required to match the topographic data within 1 foot, 
while BFEs in approximate studies can differ from the topographic data 
used by up to 20 feet and be considered compliant with FBS. 34 For the 
purpose of reporting whether FEMA has met its goal for the percentage of 
maps that are FBS compliant, a compliant detailed study counts equally 
with a compliant approximate study, with no consideration for the 
differing requirements necessary to achieve this compliance. 

In 2003, we identified linkage and clarity of measures as two key attributes 
of successful performance measures. 35 Establishing separate measures of 
compliance for detailed and approximate studies could allow FEMA to 
better use FBS compliance rates as a measure of map accuracy; however, 
the data necessary to accomplish this are presently not maintained by the 
agency. FEMA officials said that these data were not tracked at a national 
level because the significant increase in mapping activities associated with 
Map Modernization focused agency efforts on map production rather than 
data collection and analysis. FEMA officials acknowledged that the agency 
lacked a way to systematically track, at a national level, the types of 
topographic data or level of project detail used in each study, which 
limited their ability to effectively and comprehensively describe the 
accuracy of flood maps. Officials also stated that they did not consider the 
need to use data on FBS compliance rates for management decisions on 
map accuracy, as they believe that national reporting of differences in the 
level of detail used in studies does not provide significant insight into the 
flood data accuracy or reliability. Nonetheless, we continue to believe that 
FBS compliance rates reported for detailed and approximate studies 
within and across counties could provide information that would both 
enable FEMA management to compare the relative accuracy of maps, and 
be a more meaningful and understandable measure to FEMA's mapping 
stakeholders and the general public. 



For approximate studies, FEMA may include a BFE, but it not considered a regulatory 
standard. 

35 See GAO Tax Administration: IRS Needs to Further Refine Its Tax Filing Season 
Performance Measures, GAO-03-143 (Washington, D.C.: Nov. 22, 2002). GAO reported on 
nine key attributes of successful performance measures. Linkage is defined as measure that 
is aligned with division and agencywide goals and mission and clearly communicated 
throughout the organization. Clarity is defined as a measure that is clearly stated and the 
name and definition are consistent with the methodology used to calculate it. 
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While FEMA does not track this information at a national level, 
topographic data and other information regarding study detail, referred to 
as metadata, are recorded at the individual mapping project level within 
the MIP. 36 As part of its Quality Assurance process, FEMA requires 
mapping partners to submit metadata within the MIP for review. FEMA 
checks the information to ensure its validity and informs the mapping 
contractor if there are any problems that may affect map accuracy. FEMA 
reviews this metadata, but the agency does not retain or store the 
metadata in a way that enables future analysis across all mapping studies 
that could support the management of the mapping program. Standards 
for Internal Control in the Federal Government provides that agencies 
should identify, capture and distribute pertinent information in order to 
effectively carry out the agencies' duties. 37 While there is a cost associated 
with retaining and analyzing metadata, FEMA could minimize these costs 
by utilizing its existing technology, the MIP, to retain or store these data in 
a way that enables future analysis across all mapping studies. By doing so, 
FEMA could report additional information on FBS compliance and, 
thereby, have a potentially better measure of map accuracy. FEMA could 
then use this data to develop separate measures of FBS compliance for 
both detailed and approximate flood studies, each of which has a different 
range of accuracy. 

FEMA Could More Fully FEMA implemented its NVUE standard in 2007 to provide a basis for flood 

Implement Its New, Validated, mapping partners to assess the quality of new, validated, or updated 
and Updated Engineering engineering data in revising maps, but has not fully developed uniform 

Standard guidance for the validation of existing data. Validation guidance for 

mapping partners has existed in draft form since 2007, but it has not yet 
been used, according to FEMA officials. This guidance consists of a set list 
of parameters that define whether data used in the past is adequate for 
current use, or whether the area being studied has changed to an extent 
that new data is necessary. 

FEMA's draft guidance on how to validate data was found to be ineffective 
due to differing interpretations and methodologies used by various 
mapping contractors, according to FEMA officials and FEMA contractors 
who oversaw NVUE data collection and internal controls during the Map 



Metadata is a concept that applies mainly to electronically archived data and is used to 
describe the definition, structure, and administration of data files with all contents in 
context to ease the use of the captured and archived data for further use. 

37 GAO/AIMD-00-21.3.1. 
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Modernization effort. Prior to its being found ineffective, contractors used 
this guidance, issued in the form of checklists, to determine if enough 
changes had occurred in the area being studied to render existing study 
data invalid. However, two regions submitted validation figures to FEMA, 
each of which used a different methodology to obtain their calculations. 
According to FEMA, the agency determined that this process was too 
inconsistent to be acceptable as a data quality standard, so validation of 
existing data was discontinued until uniform guidance could be developed 
in 2011, leaving only new and updated data counting as NVUE compliant. 

As a result of the mid-course adjustment in 2006, FEMA set a goal of 
reaching an NVUE compliance rate of 30 percent, meaning 30 percent of 
the nation's stream miles would be mapped using new or updated 
engineering analysis by the end of Map Modernization. The goal under 
Risk MAP is to increase NVUE compliance to 80 percent to reflect this 
phase's heightened focus on ensuring data accuracy. In January 2010, 
FEMA reported the current rate of national NVUE compliance was 52 
percent. Validating existing data could assist FEMA in reaching this 
compliance goal. According to the NVUE standard, it is necessary to 
determine the relative accuracy of flood hazard data on a community's 
maps before a new mapping process begins, therefore, a needs assessment 
must be conducted to determine whether existing flood hazard 
information represents current conditions and is deemed valid or current. 
In an August 2010 report to Congress, the agency acknowledged that its 
Risk MAP strategy relies on validating the currency of a substantial 
portion of existing flood hazard information. FEMA officials said the 
development of a final version of this guidance had been a secondary 
focus as the agency was focusing its Map Modernization program 
resources on conversion of flood maps to digital format and updating the 
most significant engineering needs. Now that the initial map 
modernization program has been completed and FEMA is implementing its 
Risk MAP strategy, complete guidance for mapping stakeholders would be 
an effective and timely step to further implement its new program. 38 While 
FEMA believes that a substantial portion of existing flood hazard 
information is still current in those areas where development has not been 
significant, establishing uniform guidance for the validation of existing 
data could help FEMA ensure mapping partners are consistently validating 



Federal Emergency Management Agency, Risk Mapping, Assessment and Planning 
(Risk MAP): National Digital Elevation Acquisition and Utilization Plan for Floodplain 
Mapping, (Washington, D.C.: Aug. 9, 2010.). 
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data, and thereby help FEMA both track and report the accuracy of maps 
at the national and regional levels and better assess mapping data needs. 

FEMA's Independent FEMA has developed a quality assurance process to help ensure that 

Verification and Validation mapping efforts are performed in accordance with minimum data quality 

Process Helps Ensure Maps standards, procedures, and requirements, including independent 
Meet Minimum Requirements, verification and validation (IV&V) audits of a sample of FEMA's completed 
but the Process Could Be mapping projects. However, we identified problems with this process, 

Improved to Better Ensure which could impede FEMA's ability to ensure mapping efforts are 

Compliance and Data performed in accordance with requirements in FEMA's Guidelines & 

Reliability Specifications. The use of verification and validation is a recognized key 

practice for large and complex system development and acquisition 
projects. The purpose of the verification and validation function is to 
provide management with objective insight into the program's processes 
and associated work products. For example, IV&V audits can help FEMA 
identify problems related to compliance with data quality standards. 
However, the number of flood studies selected annually for IV&V is not 
based on probability sampling that would allow the results of the audits to 
be generalized to a larger population and used for quality assurance 
purposes. In addition, FEMA officials said the agency planned to transfer 
responsibility for the IV&V process to its program management contractor. 
However, the transfer of the responsibilities could create a potential 
conflict of interest because the program management contractor will be 
monitoring the results of its own program management efforts. Finally, the 
manner in which problems related to compliance with data quality 
standards has been documented in IV&V audit reports does not facilitate 
systematic analysis that could further enhance quality management 
efforts. 

According to the IV&V auditor, its monthly reviews were based on a 
nonprobability sample of map studies and it did not take steps to ensure 
that the audit results would be generalizable to the entire population of 
map studies. Specifically, FEMA officials told us that although the IV&V 
contractor reported in its April 2010 report 39 that map products are 
frequently not meeting FEMA's Guidelines and Specifications, FEMA 
regards these as minor issues in the agency's overall quality assurance 
framework because the IV&V contractor based their conclusions on a 
small sample of map studies (i.e., a nonprobability sample). The major 
limitation of nonprobability sampling is that the results cannot be 



IV&V Is Not Based on 
Probability Sampling 



Apptis, IV&V Audit Report (April 2010). 
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generalized to a larger population, because some members of the 
population being studied have no chance or an unknown chance of being 
selected as part of the sample. However, if FEMA's IV&V auditor used 
probability sampling, FEMA program officials would have been better 
positioned to know whether the audit issues were isolated events or 
indicative of more systemic issues in its flood mapping efforts. 

We recognize that conducting probability samples of map studies could 
involve additional costs. However, not conducting IV&V audits on a 
generalizable sample could also be costly. This is because using a 
generalizable sample could better position FEMA to identify and resolve 
systemic issues in flood mapping efforts, which is a critical task in helping 
to ensure that future efforts are performed in accordance with FEMA's 
standards and quality assurance management plans. FEMA officials stated 
that the terms of work for a new IV&V audit contractor had not yet been 
finalized and that while the officials had not determined whether the 
benefits outweigh the costs of conducting probability samples, they felt 
that reviewing the results from probability samples would be beneficial. 
Implementing probability samples in its IV&V audit process, to the extent 
that the benefits outweigh the costs, could help FEMA management use 
the results from its IV&V auditing process more strategically. 

Transfer of IV&V Audits of FEMA's mapping contractors' efforts have been conducted since 

Responsibilities to Program 2006 by an independent verification contractor; however, FEMA officials 
Management Contractor said they planned to transfer responsibility for the IV&V process to its 

Creates Potential Conflict of program management contractor by the end of this year, which will then 
Interest monitor the FEMA's mapping contractors. The transfer of these 

responsibilities creates a potential conflict of interest because the program 
management contractor is to monitor the results of its program 
management efforts. FEMA officials said they integrated the verification 
and validation process into its program management contract because the 
current IV&V contract was expiring and they believed that using FEMA's 
program management contractor for the product quality management 
would be the most effective and efficient approach for an integrated 
quality management program. FEMA officials did not believe the revised 
approach limited the program management contractor's independence or 
presented a conflict of interest. FEMA's quality management plan called 
for independent verification and validation of activities of the Program 
Manager as well as the mapping contractors. FEMA officials stated that, in 
situations where program activities of the program management 
contractor are to audited, FEMA officials would either perform the audit 
or hire an external auditor. Nevertheless, as we recently reported, the 
independence of the verification and validation contractor is a key 
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component of a reliable verification and validation function. 40 According to 
industry best practices, the verification and validation activity should be 
independent of the project and report directly to senior management to 
provide added assurance that reported results on the project's status are 
unbiased. An effective verification and validation review process should 
provide an objective assessment to management. The verification and 
validation reports should identify to senior management the issues or 
weaknesses that increase the risks associated with the project or portfolio 
so that they can be promptly addressed. FEMA management has correctly 
recognized the importance of such a function; however, the performance 
of the verification and validation function by an entity that is technically, 
managerially, and financially independent of the organization in charge of 
what it is assessing could better position FEMA to help ensure the 
independence of the verification and validation function, both in 
appearance and in fact. 

Documentation of Compliance Under FEMA's Independent Verification and Validation audit process, the 
Problems Does Not Facilitate IV&V auditor is not required to present its findings in a format readily 
Analysis conducive for performance monitoring and data analysis. For example, 

according to one PTS contractor, FEMA advised its mapping contractors 
that the IV&V audit findings are informational, rather than actionable; 
therefore, the contractors are not required to implement or track any 
changes. However, we found that at least one of the PTS contractors does 
have a system for addressing corrective action based on the IV&V audit 
findings — a corrective action process that is documented in the 
contractor's Quality Management Plan that it provided to FEMA. The 
corrective action process is used to address deficiencies identified by the 
IV&V auditor and prevent future occurrences during the mapping process, 
which are reported to FEMA through a quarterly internal quality audit 
report. 

Standards for Internal Control in the Federal Government states that 
monitoring should assess the quality of performance over time and ensure 
that the findings of audits and other reviews are promptly resolved. 41 In 
addition, we have previously reported that when agencies lacked 
systematic analysis and reporting of data, it adversely affected their ability 



GAO, Financial Management Systems: DHS Faces Challenges to Successfully 
Consolidating its Existing Disparate Systems, GAO-10-210T (Washington, D.C.: Oct. 29, 
2009). 

41 GAO, GAO/AIMD-00-21.3.1. 
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to provide complete information on the results of their operations. 42 We 
reported that this type of information could be useful to better understand 
the nature of a problem, to help plan ways to address it, and to assess 
progress made. In our analysis, we found that FEMA has several 
opportunities for improving quality outcomes using its current practices. 
First, FEMA could devise its own systematic data collection framework 
for the audits. Second, FEMA could provide more guidance to the IV&V 
auditor on how to present the audit findings. The IV&V auditor reported 
that FEMA provided no guidance on how to present the results of the 
monthly audits. FEMA could also require more comprehensive reporting 
as part of its agreement with the mapping contractors, similar to the 
internal efforts of the mapping contractor described above. The IV&V 
auditing process could include all three mapping contractors and relevant 
Cooperating Technical Partners, and FEMA could provide similar 
guidelines for reporting metrics. The IV&V audits collectively produce data 
that could be used to enhance FEMA's quality management if the 
information is leveraged properly. For example, a database of audit 
findings that is readily searchable could be used to identify trends, 
quantity recurring problems, and potentially isolate mapping issues to a 
specific region or PTS contractor. Therefore, in the absence of systematic 
data reporting, FEMA's ability to establish a corrective action plan to 
resolve issues, one of the key requirements of its quality assurance 
management program, is greatly diminished. FEMA officials stated that 
they have not required systematic data reporting of IV&V audit results 
because they viewed the findings as isolated cases to find individual map 
irregularities to assist regions in improving map accuracy rather than 
potentially systemic issues. However, FEMA officials agreed with our 
assessment that a methodical approach to IV&V data collection could 
allow the agency to better track map quality issues, better analyze the 
data, and more easily adopt a corrective action plan. These actions could 
ensure that FEMA adhere to its quality management plan and enhance 
map quality. 



" GAO, Firearms Trafficking: U.S. Efforts to Combat Arms Trafficking to Mexico Face 
Planning and Coordination Challenges, GAO-09-709 (Washington, D.C.: June 18, 2009). 
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FEMA Has Taken 
Actions to Improve 
Outreach Efforts but 
Could Enhance Its 
Efforts to Improve 
Awareness and 
Promote Map 
Acceptance 

FEMA has taken steps to increase the accessibility of outreach toolkits 
and the awareness of outreach practices, through the Internet and 
internet-based social media tools, to better equip state and local officials 
with the resources needed to effectively reach out to the public regarding 
flood mapping. FEMA previously developed and distributed outreach 
toolkits for state and local officials at the regional level by regional 
contractors. Under Risk MAP, FEMA's program management contractor is 
developing standardized outreach toolkits for state and local officials, 
which FEMA plans to provide nationally. FEMA is also developing a 
Lessons Learned Library and a secure Web site for flood mapping 
partners. As the outreach toolkits that include standardized information 
and templates are developed, the site is designed to share resources 
among FEMA regions and with state and local officials. FEMA officials 
said that the site also includes information from national and regional 
conferences that can be used by FEMA Regions and state and local 
officials to conduct outreach to the public. According to FEMA officials, 
FEMA Regions will use the secure Web site to upload examples of their 
key practices and associated materials. 

In addition to better equipping state and local officials with the resources 
needed to effectively reach out to the public, FEMA is also conducting 
outreach directly to the public. Specifically, while FEMA's regulatory and 
statutory flood mapping outreach requirements focus on notifying the 
public about flood mapping through newspaper publications and Federal 
Register notices, FEMA officials said they were considering the use of 



FEMA Is Developing 
Toolkits and a Lessons 
Learned Library for State 
and Local Mapping 
Stakeholders and Intends 
to Use Social Media Tools 
to Reach Out to the Public 
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social media sites as well. 43 FEMA has been engaging in internet-based 
social media tools and Web sites nationwide as part of its mission to 
prepare the nation for disasters. FEMA uses these tools — such as 
national-level news feeds that provide subscribers with automated 
updated information and a multimedia site that hosts videos, podcasts, 
photos and text-based documents — for flood mapping outreach efforts as 
part of their outreach strategy. 

FEMA's Risk MAP goals with social media include providing timely and 
accurate information related to disaster preparedness response and 
recovery and providing the public with another avenue for insight into the 
agency's operations. In addition, FEMA's use of social media provides 
additional outreach and channels for input. According to FEMA, citizens 
can engage more easily with the emergency management community 
through social media sites, and increase their role in disaster 
preparedness, response, and recovery. For example, FEMA has been using 
Twitter since October 2008 as a means to offer information about the 
agency's mission, efforts, and perspective. The agency also launched a 
YouTube page in October 2008 to provide stories about how its programs 
work in communities nationwide as they prepare for, respond to, and 
recover from disasters. FEMA believes that these tools could help the 
agency and its state and local mapping partners to more effectively 
communicate with communities about flood mapping efforts. 



FEMA Does Not Maintain In our review of a random sample of files containing documentation of 
Most Required Public public notification efforts, we found that FEMA does not maintain the 

Notification required documentation for public notification. Further, FEMA does not 

Documentation have a process in place to ensure that its mapping partners consistently 

±. i u document their actions to notify the public. FEMA is required by law to 

document certain actions taken to notify the public regarding the status of 
its flood mapping efforts. FEMA has requirements in place for mapping 
partners to provide such required documentation, but it does not have a 
process in place to ensure that mapping partners are meeting these 
documentation requirements. As a result, FEMA cannot be reasonably 
assured that it is complying with public notification regulations. 



USA.gov introduces Web 2.0 and social media as umbrella terms that define the various 
activities that integrate technology, social interaction, and content creation. USA.gov cites 
FEMA as an example of an agency that is using online content and technology to achieve 
its mission and goals. 
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FEMA is required to maintain a Flood Elevation Determination Docket 
(FEDD) file for every local government that is affected by a flood mapping 
project that results in a change in base flood elevation. 44 The FEDD file 
provides a record of all matters pertaining to flood elevation 
determinations, including public notification requirements established by 
the National Flood Insurance Act of 1968. These FEDD files are required 
to contain documentation demonstrating that a mapping contractor took 
the following six public notification actions: 

• notifying the community's CEO of the proposed flood elevation 
determination, 

• notifying the public of the proposed flood elevation determination via 
an initial newspaper publication, 

• notifying the public of the proposed flood elevation determination via a 
2nd newspaper publication, 

• notifying the public of the proposed flood elevation determination via a 
notice in the Federal Register, 

• notifying the public of the final flood elevation determination via a 
notice in the Federal Register, and 

• notifying the community's CEO of the final flood elevation 
determination. 

Based on our file review of a random sample of counties with flood 
mapping projects, FEMA did not have a FEDD folder on file for 
approximately 67 percent of the counties that had completed mapping 
projects since 2005. 45 We estimate that FEMA complied with some, but not 
all documentation requirements for 16 percent of the counties, and 
complied with all 6 documentation requirements for the remaining 17 
percent, as illustrated in figure 6 below. Because FEMA does not 
sufficiently maintain documentation of its public notification activities, the 
FEDD files do not provide a means for the agency to provide reasonable 
assurance that it is complying with public notification regulations. As a 
result, FEMA cannot use the FEDD files to determine the extent to which 
a community was notified about new mapping projects in accordance with 
the six public notification actions. 



1 44 C.F.R. § 67.3. 
' The maximum n 

plus or minus 9 percentage points at the 95 percent level of statistical confidence. 



45 The maximum margin of error for estimates of percentages from our sample counties is 
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Figure 6: Estimated Compliance Rates with FEMA Documentation Regulations: 
Counties Having Flood Mapping Projects Since 2005 That Resulted in a Change in 
Base Flood Elevation 




1 Compliance cannot be determined; FEDD 

file not found 

Source: GAO analysis of FEMA data. 



FEMA officials said that they rely on mapping partners to document 
completion of public notification requirements in FEDD files and FEMA 
provides mapping partners with background on public notification 
documentation requirements. FEMA also directs mapping partners to 
document compliance with notification requirements in FEMA's 
Document Control Procedures Manual. In reviewing FEMA's manual, we 
determined that if mapping contractors followed the guidance in the 
manual, they should be able to comply with public notification 
documentation requirements. FEMA's manual provides details on the 
procedures to be followed and the documents to be used for each NFIP 
map action including FEMA-initiated, FEMA-contracted, and community- 
initiated map studies and revisions. 

FEMA relies on mapping partners to comply with these requirements, but 
the agency does not have a process in place to ensure that these mapping 
partners consistently document their actions to notify the public. FEMA 
regulations require that public notification documentation reside in FEDD 
files, which we observed as part of our file review, as shown in figure 7 
below. 
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Figure 7: GAO Analyst Examining Files at FEMA's Engineering Library 




Source: GAO. 



FEMA's contractor oversees its Engineering Library, where the hard copy 
FEDD files are maintained. FEMA officials said that the agency is aware 
that its mapping partners are not complying with public notification 
documentation requirements and is drafting a procedural memorandum 
that reiterates the processes that mapping partners are to comply with to 
ensure that all documentation from completed studies are sent to FEMA in 
a timely manner. 

Standards for Internal Control in the Federal Government states that 
controls should generally be designed to assure that ongoing monitoring 
occurs in the course of normal operations. This may include regular 
management and supervisory activities, comparisons, reconciliations, and 
other actions to ensure compliance with applicable laws and regulations. 
FEMA established its Mapping Information Platform (MIP) to manage the 
mapping production process, and the system includes data fields for each 
of the public notification requirements, but the agency does not use the 
information in the MIP to document compliance. Mapping partners have 
used the MIP to document four of the six public notification requirements 
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(related to publications in the local newspaper for preliminary maps and 
the Federal Register for preliminary and final maps). However FEMA 
management does not have access to the two data fields in the MIP related 
to requirements to notify the community's CEO of the proposed flood 
elevation determination and of the final flood elevation determination. 
FEMA officials said that these two fields could be made accessible so that 
all six public notification requirements could be documented by mapping 
partners in the MIP. FEMA officials said that they do not use the MIP to 
document compliance with public notification requirements because the 
FEDD folder is the official record maintained for that purpose. A 
mechanism to monitor compliance with public notification documentation 
regulations and statutes could help FEMA obtain reasonable assurance 
that its contractors are complying with documentation requirements, 
which can help FEMA ensure that the public is being notified as required. 



FEMA Could Better Assess 
the Effectiveness of Its 
Outreach Efforts 



FEMA is collecting some data on the quantity and nature of appeals to the 
mapping studies, but the agency is not capturing data on all appeals and 
protests in a manner that could be used to inform its decisions about 
where to focus outreach efforts. Before flood maps that result in a change 
in base flood elevation become effective, regulations require that FEMA 
hold a 90-day appeals period. 46 Appeals and protests can be submitted by 
state and local officials or by individual members of the public. Appeals 
challenge the proposed base flood elevation based on technical or 
scientific inaccuracy, while all other challenges to the flood maps are 
treated as protests. State and local officials are responsible for collecting 
and reviewing all individual appeals and protests and forwarding them to 
FEMA. An official may also submit an appeal on behalf of the local 
government itself. Figure 8 below provides an overview of the flood 
mapping process and where the appeals period occurs in the process. 



46 44 C.F.R. § 67.5. 
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Figure 8: Overview of Outreach Steps in the Flood Mapping Process 




Sources: GAO analysis of FEMA data and Art Explosion clipart. 



FEMA divides appeals into two groups — eligible and ineligible. FEMA 
defines eligible appeals as appeals from communities or individuals which 
are based on knowledge or information indicating that the elevations 
proposed by FEMA are scientifically or technically inaccurate and that 
contain supporting documentation. To qualify as eligible appeals, requests 
must also be submitted to FEMA within the 90-day appeal period. FEMA 
defines ineligible appeals as appeals that do not meet these 
requirements — for example, if an appeal is based on something other than 
the scientific or technical accuracy of the elevations, or does not include 
supporting documentation. 

During our analysis of the MIP, we found that FEMA does not broadly 
capture information on the appeals and protest that can be used to analyze 
trends nationally. For example, FEMA does not use the MIP to track 
ineligible appeals or protests from local communities or individuals, even 
though these appeals and protests may reflect state and local officials' or 
the public's disagreement with a flood map. Rather, when FEMA receives 
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ineligible appeals or protests, it combines the data in the MIP with 
comments that are received from state and local officials prior to the end 
of the appeal period. FEMA then labels all of the ineligible appeals, 
protests, and comments as protests, even though ineligible appeals data 
could serve as an indicator of the public's acceptance of, or resistance to, 
flood maps. FEMA officials stated that they do not comprehensively 
collect and analyze data on appeals and protests, as FEMA uses 
information in the MIP for project-by-project supervision of flood map 
studies, rather than as a strategic management tool to analyze trends. Also, 
FEMA has not yet considered the costs and benefits of such analyses. We 
have previously reported that in order to monitor progress, performance 
data should be gathered to determine how well performance goals are 
being achieved. 47 While we recognize there is a cost associated with 
collecting and analyzing all data on appeals and protests, should FEMA 
determine that the benefits outweigh the costs, taking such action could 
help FEMA evaluate the extent to which public acceptance is being 
achieved and better target outreach activities to more resistant 
communities. 

In addition, FEMA has not established guidance for how to collect and 
review appeals and protests, nor has it established guidance on how to 
report appeals and protests data in the MIP. Instead, flood community 
officials are directed to collect the appeals and protest data, review them, 
and decide which ones to submit to FEMA. Once FEMA receives the 
appeals and protests, it reviews them to determine whether they are 
eligible. If any issues arise or if additional supporting data is needed, 
FEMA will work with community officials, mapping contractors, or other 
mapping partners to acquire it. FEMA resolves appeals using the 
documentation originally submitted or it will consider additional data or 
supporting information if supplied generally within 30 days. 

However, in the absence of guidance on a standardized process, state and 
local officials may collect and review appeals and protests in different 
manners from one another, or from FEMA, and FEMA officials stated that 
they do not know what process any one contractor used to make its 
decisions about which appeals and protests to refer to FEMA. For 
example, in the case of Hillsborough County, Florida, county officials 
stated that they received over 1,000 appeals and protests. The county hired 



' GAO, 2010 Census: Cost and Design Issues Need to Be Addressed Soon, GAO-04-37 
(Washington, D.C.: Jan. 15, 2004). 
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a contractor to review the appeals and protests the county received; the 
contractor consolidated them into approximately 400 appeals and 
protests, and responded to the remainder at the local level, according to 
the contractor. After the contractor's review and recommendation, the 
county submitted approximately 150 appeals and protests to FEMA for 
review and disposition. FEMA officials then aggregated these appeals and 
protests to input into the MIP, according to FEMA officials. MIP records 
indicate that Hillsborough County, Florida, has 60 appeals and protests 
from this study. Thus, more than 1,000 appeals and protests were 
condensed to a fraction of that total, and FEMA has no way of knowing 
what criteria its contractor used when making decisions about which 
appeals and protests it submitted to FEMA. FEMA officials acknowledged 
that they have not established a standardized process for contractors on 
how to enter appeals and protest data. 

In 2009, FEMA identified appeals and disputes arising from the study and 
mapping process as a concern of mapping partners, and requested the 
Association of State Floodplain Managers to review FEMA's mapping 
processes to identify ways of improving the quality and effectiveness of 
FEMA's communications with state and local officials prior to and during 
the floodplain study and mapping process. This study is still ongoing and is 
expected to be completed by the end of year 20 10. 48 Standards for 
Internal Control in the Federal Government provides that agencies to 
establish policies and procedures, techniques and mechanisms to enforce 
management's directives. Providing guidance to standardize the process 
mapping partners use to make decisions about which appeals and protests 
to submit to FEMA could help the agency better ensure it has complete 
information on appeals and protests in each community. 



In addition to contracting with ASFPM to review appeals process, FEMA officials said 
that, beginning in November 2010, they planned to implement an additional process for 
appeals resolution. Specifically, FEMA has created an independent Scientific Resolution 
Panel that can be convened when deemed necessary by FEMA or by a joint agreement 
between FEMA and an appellant. The Scientific Resolution Panel is to review and resolve 
conflicting data related to proposed BFEs as provided for in the National Flood Insurance 
Act, as amended. 
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FEMA has not identified performance goals for public acceptance of flood 
maps and has not developed measures to evaluate the extent to which it 
would achieve these goals. FEMA's Risk MAP National Outreach Strategy 
includes objectives for achieving 5 percent increases in the publics' and 
local officials' awareness and understanding of flood risk and future 
vulnerability to flooding by fiscal year 2011. However, this strategy does 
not include a performance goal and performance measures for the public's 
acceptance of flood maps. While we recognize that developing measures 
to gauge public acceptance of flood maps is not easy, it is possible to 
develop indicators of public resistance to flood maps. For example, the 
volume of appeals and protests of a particular mapping study could be an 
indicator of public resistance to flood maps. Thus, FEMA could develop a 
performance goal related to increasing public acceptance of flood maps 
and could use ineligible appeals as an indicator of the public's resistance 
of flood maps, which could serve as a performance measure for this goal. 
For example, assuming that a certain percentage of individuals will be 
opposed to a map if it requires them to purchase flood insurance, FEMA 
could determine an expected rate of ineligible appeals and use this as a 
baseline measure. 

Our past work on the experience of leading organizations has 
demonstrated that the principles of establishing measurable goals and 
related measures, developing strategies for achieving results, and 
identifying the resources that will be required to achieve the goals are the 
basic underpinning for performance-based management — a means to 
strengthen program performance. 49 FEMA officials stated that they have 
considered developing performance goals and measures related to public 
acceptance of maps, but have not taken any action to date. According to 
FEMA officials, the agency does not currently collect information on 
outreach activities because it is not within the scope of the Risk MAP 
Quality Assurance Management Plan. While this plan identifies quality 
standards and metrics for outreach as potential future scope, it does not 
identify in what years such outreach metrics are to be addressed. 
Developing performance goals and measures for public acceptance of 



For example, see GAO, Managing for Results: Enhancing Agency Use of Performance 
Information for Management Decision Making, GAO-05-927 (Washington, D.C.: Sept. 9, 
2005); GAO, Program Evaluation: Studies Helped Agencies Measure or Explain Program 
Performance, GAO/GGD-00-204 (Washington, D.C.: Sept. 29, 2000); GAO, Agency 
Performance Plans: Examples of Practices That Can Improve Usefulness to 
Decisionmakers, GAO/GGD/AIMD-99-69 (Washington, D.C.: Feb. 26, 1999); and GAO, 
Managing for Results: Strengthening Regulatory Agencies ' Performance Management 
Practices, GAO/GGD-00-10 (Washington, D.C.: Oct. 28, 1999). 
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Acceptance of Flood Maps 
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flood maps could help FEMA better determine whether its outreach efforts 
are achieving their intended results. 



FEMA Has Not 
Determined What 
Resources Are Needed for 
Outreach and Has Not 
Established Risk-Based 
Mapping Priorities for 
Outreach Activities 



FEMA has not determined the financial or human resources that are 
necessary to conduct flood mapping outreach efforts. FEMA officials, and 
the state and local officials we spoke with in all four mapping project we 
reviewed said that they devote most of their resources to map production. 
As a result, outreach activities that could lead to increased map 
acceptance may be under-resourced because map accuracy is a higher 
priority for FEMA and its mapping partners. For example, FEMA does not 
have a line-item in its budget for flood mapping outreach, and agency 
officials said that outreach staffing costs are paid out of general NFIP 
funds, and are not individually tracked. In addition, FEMA is unable to 
analyze outreach spending within the CTP program as a whole. However, 
while FEMA is able to track total state and local contributions under their 
CTP contracts, 50 FEMA is unable to specifically track the amount of 
funding going toward outreach versus other mapping activities, according 
to FEMA officials. 



FEMA's Risk MAP strategy states as one of its goals to improve the 
utilization of resources, but the agency cannot determine this if it does not 
track the resources it is devoting to its various activities. Furthermore, a 
key purpose of the Government Performance and Results Act 51 is to create 
closer and clearer links between the process of allocating scarce resources 
and the expected results to be achieved with those resources. 52 FEMA 
officials stated that annual budgeting for its flood mapping activities is 
allocated across FEMA regions based on the regions' level of mapping 
activities and by FEMA's risk based strategy, and it is up to each region to 
identify the outreach personnel and resources needed within their regional 
mapping budgets. However, FEMA officials could not provide us with 
budget or expenditure information on outreach activities at FEMA regions 



According to the CTP program guidance, if a CTP has agreed to perform outreach 
activities, FEMA may match funds from CTPs for outreach activities up to 10 percent. 

51 Pub. L. No. 103-62, 107 Stat. 285 (1993). 

52 GAO, Performance Budgeting: OMB's Performance Rating Tool Presents Opportunities 
and Challenges For Evaluating Program Performance, GAO-04-550T (Washington, D.C.: 
Mar. 11, 2004). 
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because FEMA has not established a reporting structure with which the 
regions can provide that information to it. 

In addition, while FEMA has developed its Risk MAP outreach strategy, it 
has not developed a risk-based approach for conducting outreach 
activities for flood mapping that could enable it to target resources 
effectively. In 2004, as part of Map Modernization, FEMA established risk- 
based mapping priorities by ranking all 3,146 counties from highest to 
lowest for risk of flooding based on a number of factors, including 
population, growth trends, housing units, flood insurance policies and 
claims, repetitive loss properties, and flood disasters. FEMA has not 
incorporated these risk-based priorities into its Risk MAP outreach 
strategy. FEMA officials stated that risk class is considered in an effort to 
anticipate possible outreach needs on a case-by-case basis. According to 
FEMA officials, certain issues will trigger a greater emphasis on outreach 
efforts. For example, as result of an increased focus on protection 
provided by levees since 2005, some communities — historically protected 
by levees — are now considered to be Special Flood Hazard Areas and 
subject to mandatory purchase of flood insurance. 53 While FEMA's 
approach considers these issues on a case-by-case basis, its response to 
events is reactive to these events and does not enable FEMA to 
systematically plan and budget its resources more efficiently and 
effectively. We reported in December 2005, that risk management is a 
widely endorsed strategy for helping policymakers make decisions about 
allocating finite resources. 54 By providing a reporting structure for regions 
to identify and justify their outreach resource needs, FEMA could better 
plan for and report on specific outreach activities for flood mapping on a 
regional or national level. Likewise, by using risk in its decisions regarding 
the allocation of outreach resources — for example, by considering the 
number of high risk counties or the number of mapping projects under 
way in a particular region — FEMA could ensure that its resources for flood 
mapping outreach efforts are allocated in the most effective manner. 



In 2011, we plan to respond to a congressional mandate to review FEMA's management 
of national levee systems. 

54 GAO, Risk Management: Further Refinements Needed to Assess Risks and Prioritize 
Protective Measures at Ports and Other Critical Infrastructure, GAO-06-91 (Washington, 
D.C.: Dec. 15, 2006). 
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FEMA Does Not Leverage 
FloodSmart Marketing 
Resources to Enhance Its 
Outreach Efforts 



FEMA does not leverage its existing resources by using NFIP FloodSmart 
marketing resources to enhance its flood mapping outreach efforts. FEMA 
has three divisions in its Mitigation Directorate that share roles and 
responsibilities in conducting outreach: the Risk Analysis Division, the 
Risk Insurance Division, and the Risk Reduction Division. The Risk 
Insurance Division is responsible for a marketing effort called FloodSmart. 
According to FEMA, FloodSmart is a national integrated marketing 
campaign that utilizes mail, television, internet, and print media as 
marketing tools to promote the purchase of flood insurance policies. 

One mechanism FloodSmart uses to help market flood insurance is 
through direct mail to the public. According to FEMA officials, to support 
insurance agents in talking to their clients about flood insurance, 
FloodSmart provides insurance agents access to a Mail-On-Demand 
program through a Web site. The Mail-On-Demand program includes a 
direct mail template that informs property owners about proposed map 
changes in their community and how those changes may affect their flood 
insurance needs. The Mail-On-Demand program allows insurance agents to 
access a list of potential flood insurance purchasers. In addition, this Web 
site contains a schedule that shows the preliminary date, appeals dates, 
Letter of Final Determination date, and effective dates for communities 
undergoing a mapping study. 

FEMA FloodSmart officials said they also had developed two "toolkits" of 
media materials that FloodSmart employees may provide to state and local 
officials, insurance agents, and other stakeholders. One of the toolkits is 
designed to provide information on the mapping process, and the second 
toolkit provides information on levee safety and certification and the 
effect of levees on FEMA flood maps. FEMA Floodsmart officials said 
these media kits include general templates of informational materials that 
can be customized for specific areas and used to communicate the 
importance of flood insurance surrounding map and levee changes to 
communities where the flood risk designation has changed as a result of 
an assessment or flood mapping efforts. According to FEMA officials, they 
created the map change toolkit materials in partnership with Hillsborough 
County, Florida, and the levee toolkit was developed in conjunction with 
the Sacramento Flood Control Agency. They said they share the toolkits at 
industry and stakeholder conferences, mail them out in response to 
specific requests, and post the materials on FloodSmart.gov. FEMA's 
FloodSmart marketing efforts are initiated at the request of a FEMA 
region. FloodSmart team members said that, in those instances, they 
typically do not send out direct mailers regarding the importance of flood 
insurance until right before maps become effective. FEMA officials stated 
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that the Risk MAP program's outreach strategy is promoting greater 
coordination and regular meetings between FEMA's FloodSmart team, 
flood mapping staff, and FEMA staff responsible for floodplain 
management. 

FloodSmart marketing efforts could be used by FEMA in the process to 
help promote community awareness, education, and acceptance of flood 
maps. We have previously reported that collaborative efforts are enhanced 
when agencies identify and address needs by leveraging resources to 
support a common outcome. 55 In this case, FEMA's Risk Analysis Division 
and its Risk Insurance Division could enhance their collaboration by 
applying this practice. Given that FloodSmart already has efforts under 
way to help to educate the public on the potential flood risk in 
communities and to encourage them to take action, these efforts could be 
targeted toward educating the public about, and encouraging public 
acceptance of FEMA's flood mapping efforts. While FEMA officials stated 
that the Risk MAP outreach strategy is promoting greater coordination 
with FEMA's FloodSmart team, FEMA could enhance its flood mapping 
outreach efforts by leveraging FloodSmart's marketing resources and 
expertise to increase public acceptance of flood maps. 



The results of the flood mapping process on individual property owners 
subject to resulting flood insurance requirements can be significant. To 
effectively implement FEMA's 5-year Risk MAP program goal of improving 
the accuracy of flood maps, FEMA will need to continue to improve its 
data standards and its management processes. Since federal law requires 
FEMA to assess the need to revise and update the nation's flood maps at 
least every 5 years, determining how best to use mapping resources will be 
crucial. Establishing separate measures of compliance for detailed and 
approximate studies could allow FEMA to better use FBS compliance 
rates as a measure of map accuracy; however, the data necessary to 
accomplish this are presently not maintained by the agency. By retaining 
and analyzing metadata, FEMA could report additional information on 
FBS compliance and, thereby, have a potentially better measure of map 
accuracy. Further, FEMA's NVUE standard provides a basis for flood 
mapping partners to assess the quality of new, validated, or updated 
engineering data in revising maps; however, establishing uniform guidance 



00 GAO, Results-Oriented Government: Practices That Can Help Enhance and Sustain 
Collaboration among Federal Agencies, GAO-06-15 (Washington, D.C.: Oct. 21, 2005). 
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for the validation of existing data could help FEMA ensure mapping 
partners are consistently validating data. This step could help FEMA both 
track and report the accuracy of maps at the national and regional levels 
and better assess mapping data needs. In addition, FEMA's IV&V process 
helps ensure that mapping efforts are performed in accordance with 
minimum data quality standards, procedures, and requirements. However, 
implementing probability sampling during the IV&V auditing process, to 
the extent that the benefits outweigh the costs, would ensure that the 
results are generalizable and could help FEMA management use the 
information more strategically. Likewise, to maintain the independence, 
both in appearance and fact, of FEMA's verification and validation 
function, this auditing function should be performed by an entity that is 
technically, managerially, and financially independent of the organization 
in charge of what is being assessed. And finally, in the absence of 
systematic data reporting, FEMA's ability to establish a corrective action 
plan to resolve issues identified through the IV&V process, one of the key 
requirements of its quality assurance management program, is greatly 
diminished. 

Regarding outreach, FEMA has taken positive actions regarding its 
innovative use of new media to enhance its outreach efforts. However, 
there are areas in which FEMA could enhance its outreach efforts. For 
example, without a mechanism to monitor mapping contractors' 
compliance with public notification documentation requirements, FEMA is 
limited in its ability to provide reasonable assurance that the agency is 
notifying the public as required. Opportunities also exist for FEMA to 
better utilize data on community appeals and protests to inform its 
decisions about where to focus outreach efforts, and provide guidance to 
standardize the process by which mapping partners analyze appeals and 
protests data to the extent that the benefits outweigh the costs. Moreover, 
without specific performance goals and measures to assess the 
effectiveness of outreach efforts related to flood mapping, it may be 
difficult for FEMA to determine whether its outreach efforts are achieving 
their intended results. Further, by providing a reporting structure for 
regions to identify and justify their outreach resource needs, FEMA could 
better plan for and report on specific outreach activities for flood mapping 
on a regional or national level. Likewise, by using risk in its decisions 
regarding the allocation of outreach resources — for example, by 
considering the number of high risk counties or the number of mapping 
projects under way in a particular region — FEMA could ensure that its 
resources for flood mapping outreach efforts are allocated in the most 
effective manner. Finally, FEMA has the opportunity to leverage existing 
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resources by broadening the scope of the FloodSmart program that 
supports the NFIP to help promote public acceptance of flood maps. 



Recommendations for 
Executive Action 



• establish separate measures and collect data needed to assess 
compliance with the Floodplain Boundary Standard for detailed and 
approximate flood studies, and 

• establish uniform guidance for the validation of existing engineering 
data to help FEMA fully implement the N VUE standard and provide a 
basis for mapping partners to validate flood hazard data. 

To enhance the independent verification and validation (IV&V) audit 
process, we recommend the Administrator of the Federal Emergency 
Management Agency: 

• implement probability sampling during the IV&V audit process to the 
extent that the benefits outweigh the costs, to ensure that the results 
are generalizable for decisionmaking; and 

• transfer IV&V duties back to an independent entity to help ensure 
impartiality; and 

• adopt a systematic approach to IV&V data collection, so FEMA can 
better track map quality issues, more easily analyze the data, and adopt 
a corrective action plan. 

To address challenges in improving community outreach, we recommend 
that the Administrator of the Federal Emergency Management Agency: 

• establish a mechanism to better ensure compliance with the 
documentation requirements of public notification regulations; 

• collect and analyze data on appeals and protests, including those on 
ineligible appeals, to the extent that the benefits outweigh the costs; 

• issue guidance to mapping stakeholders to standardize the process for 
analyzing appeals and protests and submitting this data to FEMA; 

• establish performance goals and measures for promoting public 
acceptance of flood maps; 

• develop a reporting structure for regions to use to identify resources 
needed to conduct flood mapping outreach activities, and implement a 
risk-based approach to allocate outreach resources; and 



We are making 11 overall recommendations. 

To address challenges in ensuring the accuracy of flood maps, we 
recommend that the Administrator of the Federal Emergency Management 
Agency: 
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• leverage, as appropriate, existing FloodSmart marketing resources and 
expertise to help increase public acceptance of flood maps. 



Agency Comments 
and Our Evaluation 



We provided a draft of this report to DHS for review and comment. DHS 
provided written comments, which are reprinted in appendix IV. DHS also 
provided us with technical comments, which we considered and 
incorporated as appropriate. We also provided a draft of this report to the 
U.S. Army Corps of Engineers, but it did not provide written comments on 
the report. However, in an email on November 3, 2010, the Corps of 
Engineers liaison indicated that the Corps had no comments on the report. 



In commenting on the draft report, DHS stated that it concurred with 10 of 
the 11 recommendations. For the recommendations for which DHS 
concurred, the agency identified actions taken or plans to implement 
them. Specifically, FEMA agreed with our recommendation to establish 
tracking and reporting that will allow FEMA to report the level of 
precision by flood source. FEMA plans to implement this tracking as part 
of the Coordinated Needs Management System that will be fully 
implemented this fiscal year. FEMA agreed with our recommendation and 
will finalize and issue uniform guidance for the validation of existing 
engineering data under its New, Validated, or Updated Engineering 
(NVUE) standard in this fiscal year. FEMA also agreed to assess the 
additional costs and expected benefits to expand the scope of the IV&V, 
and to implement probability sampling and will work with the contractor 
performing audits to redesign the reporting of map quality issues. FEMA 
stated it will issue guidance to address our recommendations to ensure 
compliance with the documentation requirements and to standardize the 
process for submitting appeals and protests data to FEMA in order to 
improve the collection and analysis of this information. FEMA also stated 
it will explore potential new measures and look for ways to quantify and 
track outreach activities that are integrated into map production activities. 
In addition, FEMA also concurred with our recommendation to leverage 
existing FloodSmart marketing resources, stating that FEMA's flood 
mapping program is working with its FloodSmart program as part of the 
Risk MAP program to identify areas where outreach effectiveness can be 
increased and consistent messages can be delivered. These actions should 
address our recommendations and help FEMA improve its efforts to 
ensure flood map accuracy and enhance the agency's outreach efforts in 
developing and implementing new flood maps. 



DHS did not concur with our fourth recommendation that the 
Administrator of the Federal Emergency Management Agency should 
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transfer independent verification and validation duties (IV&V) back to an 
independent entity to help ensure impartiality. The department's response 
stated that FEMA's Program Manager contractor is technically, 
managerially, and financially independent of the flood hazard development 
process and that the contractor is helping FEMA to integrate the program, 
monitor program performance, and implement a quality management 
process. FEMA believes that the quality audit function is more effective if 
it is integrated into the overall quality management process rather than 
performed externally to the quality management process. However, as we 
noted in the report, according to industry best practices, the verification 
and validation activity should be independent of the project and report 
directly to senior management to provide added assurance that reported 
results on the project's status are unbiased. As FEMA states, its flood 
mapping Program Manager contractor is an integral part of FEMA's flood 
mapping program management, and as such we continue to believe that 
the program management contractor's programmatic responsibilities and 
involvement prevent it from having a clearly independent role in validating 
and verifying the results of flood map production activities, because the 
contractor has a vested interest in overall program performance. 
Therefore, we believe that FEMA should transfer independent verification 
and validation duties back to an independent entity to help ensure 
impartiality. 



We are providing copies of this report to interested congressional 
committees, the FEMA Administrator, the Secretary of Homeland Security 
and other interested parties. This report will also be available at no charge 
on the GAO Web site at http://www.gao.gov. 
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If you or your staff have any questions concerning this report, please 
contact me at (202) 512-8757 or by e-mail atjenkinswo@gao.gov. Contact 
points for our Offices of Congressional Relations and Public Affairs may 
be found on the last page of this report. Key contributors to this report are 
listed in appendix V. 




William 0. Jenkins, Jr. 
Director 

Homeland Security & Justice Issues 
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Appendix I: Objective, Scope, and 
Methodology 



This report addresses the following objectives: 

• To what extent has FEMA taken actions to enhance the accuracy of 
flood maps and what challenges, if any, does FEMA face? 

• To what extent has FEMA taken actions to help promote community 
acceptance of flood maps, and what challenges, if any, does FEMA 
face? 

We focused our review on those standards and processes related to flood 
hazard mapping for rivers and streams (commonly know as "riverine" 
flooding 1 ), as these account for about 95 percent of FEMA's flood maps. 
As a result, we limited our scope to exclude those standards and processes 
related to flood hazard mapping for coastal areas and the levee 
certification. In addition, FEMA has processes to modify and update flood 
map information during the time that a community's maps are in effect, 
called a Letter of Map Change, which is also outside the scope of our 
work. 

To evaluate the extent that FEMA has taken actions to ensure data quality 
standards are consistently met for flood maps updated since 2005 and to 
what extent has FEMA measured whether implementation of the data 
quality standards results in accurate flood maps, we analyzed information 
on FEMA's policies and plans for flood map modernization and data from 
FEMA's Mapping Information Platform (MIP) and systems for 
documenting compliance with data quality standards. Specifically, we 
reviewed documents including FEMA's Risk Mapping, Assessment and 
Planning (Risk MAP) strategy, the Risk Map Multi Year Plan, and Map 
Modernization Guidelines and Specifications for Flood Hazard Mapping 
Partners and its associate appendices. We assessed FEMA's standards and 
guidance to criteria developed in recent reports by the National 
Academies of Sciences and the National Research Council, specifically the 
Academies' report "Mapping the Zone: Improving Flood Map Accuracy. 2 
We discussed the reports' methodologies with the authors and with 
relevant FEMA officials, and analyzed reviews and critiques of the 
Academies' reports to determine that they were appropriate for our 
purposes. We tested the controls on the quality assurance /quality control 



1 Riverine flooding is flooding related to or caused by a river, stream, or tributary 
overflowing its banks due to excessive rainfall, snowmelt or ice. 

2 National Research Council (U.S.), and United States. 2009. Mapping the Zone: Improving 
Flood Map Accuracy. Washington, D.C.: National Academies Press. 
http://www.nap.edu/catalog.php?record_id= 12573 May 2009. 
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Appendix I: Objective, Scope, and 
Methodology 



(QA/QC) process by extracting and reviewing data on all projects initiated 
and completed for the period of October 2005 (corresponding to FEMA's 
Mid-Course Adjustment to its Map Modernization Initiative) through 2009. 
We also reviewed FEMA's Floodplain Boundary Standard (FBS) and New, 
Validated, or Updated Engineering (NVUE) verification systems that were 
designed to track implementation of data accuracy requirements. We 
tested the controls on the FBS and NVUE compliance process by 
extracting and reviewing data on all projects initiated and completed from 
fiscal year 2006 (when the FBS was established) through 2009 and 
compared them against criteria in Standards for Internal Control in the 
Federal Government. 3 To assess the reliability of these databases, we 
compared data to FEMA's management reports, interviewed FEMA's three 
mapping contractors and reviewed the original data generated by these 
contractors. We determined that the FBS and NVUE compliance data were 
sufficiently reliable for the purposes of this report. We also compared 
FEMA's QA/QC process to effective practices we have identified for 
quality assurance. 4 Finally, we discussed FEMA's mapping process and 
standards with agency officials, as well as officials from other federal 
stakeholders including the U.S. Geological Survey, the National Oceanic 
and Atmospheric Administration, the U.S. Army Corps of Engineers; 
national organizations including the Association of State Floodplain 
Managers, and National Association of Flood & Stormwater Management 
Agencies. We also discussed FEMA's mapping process and standards with 
agency officials, as well as officials from other federal stakeholders in 
geographic data collection and mapping including the U.S. Geological 
Survey, the National Oceanic and Atmospheric Administration, the U.S. 
Army Corps of Engineers and the National States Geographic Information 
Council; subject-matter experts on flood hazards and floodplain 
management from national organizations including the Association of 
State Floodplain Managers, and National Association of Flood & 
Stormwater Management Agencies who are stakeholders to FEMA's 
mapping initiatives, as well as state and local officials involved in mapping 
projects we selected in Arizona, Florida, California, and North Carolina (a 
discussion of the selection process is included below). 



3 GAO, Standards for Internal Control in the Federal Government, GAO/AIMD-00-21.3.1 
(Washington, D.C.: November 1999). 

4 GAO, Best Practices: Commercial Quality Assurance Practices Offer Improvements for 
DOD, NSIAD-96-162 (Washington, D.C.: Aug. 26, 1996.). 
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Methodology 



To assess the extent to which FEMA has taken actions to help promote 
community acceptance, and ensured that regulatory requirements for 
documenting public notification efforts are consistently met for flood 
maps updated since 2005 we analyzed information on FEMA's policies and 
plans for community outreach and data from FEMA's Mapping 
Information Platform and systems for documenting compliance with 
statutory and regulatory requirements for coordination with state and 
local officials involved in mapping projects. Specifically, we reviewed 
FEMA's previous Outreach Strategy for Map Modernization and its new 
Risk MAP National Outreach Strategy, and analyzed the goals and 
performance measures of FEMA's outreach strategy for Map 
Modernization, and its new Risk MAP national outreach strategy against 
our prior work reviewing federal agencies' practices for development of 
national strategies compared it to effective practices we have identified for 
national strategies, 5 as well as review FEMA's budget and staff allocations 
related to outreach. To assess FEMA's internal controls and program 
management of community outreach efforts, we examined several FEMA 
databases, including the Mapping Information Platform (MIP), discussed 
above, designed to document state and local mapping stakeholder 
information. As noted above, we tested the controls on the MIP by 
extracting and reviewing data on all projects initiated and completed from 
2005 through 2009 and assessed the reliability of these databases by 
checking them against documents, such as FEMA's management reports 
and Flood Elevation Determination Dockets (FEDD) that are established 
for each mapping project. We also interviewed FEMA, state, and local 
officials involved in flood map outreach to obtain their perspectives. For 
our review of FEMA's compliance with public notification documentation 
requirements, we examined FEMA's Flood Elevation Determination 
Dockets (FEDD) that are established for each mapping project, selecting a 
probability sample of 88 counties from a population of 431 counties that 
had completed studies since 2005, that resulted in a change in base flood 
elevation. From this probability sample we reviewed mapping partners' 
documentation of compliance with six documentation requirements 
below: 

• notifying the community's CEO of the proposed flood elevation 
determination, 



5 See GAO, Combating Terrorism: Evaluation of Selected Characteristics in National 
Strategies Related to Terrorism, GAO-04-408T (Washington, D.C.: Feb. 3, 2004), and GAO, 
Financial Literacy and Education Commission: Further Progress Needed to Ensure an 
Effective National Strategy, GAO-07-100 (Washington, D.C.: Dec. 4, 2006). 



Page 52 



GAO-11-17 FEMA Flood Maps 



Appendix I: Objective, Scope, and 
Methodology 



• notifying the public of the proposed flood elevation determination via 
an initial newspaper publication, 

• notifying the public of the proposed flood elevation determination via a 
2nd newspaper publication, 

• notifying the public of the proposed flood elevation determination via a 
notice in the Federal Register, 

• notifying the public of the final flood elevation determination via a 
notice in the Federal Register, and 

• notifying the community's CEO of the final flood elevation 
determination. 

Because we followed a probability procedure based on random selections, 
our sample is only one of a large number of samples that we might have 
drawn. Since each sample could have provided different estimates, we 
express our confidence in the precision of our particular sample's results 
as a 95 percent confidence interval (e.g., plus or minus 9 percentage 
points). This is the interval that would contain the actual population value 
for 95 percent of the samples we could have drawn. Percentage estimates 
based on our sample of counties have 95 percent confidence intervals no 
wider than +/- 9 percentage points. 

To supplement our analyses of FEMA's flood mapping internal controls 
and program management activities related to both data accuracy and 
community outreach, we selected four flood map modernization projects 
in Arizona, Florida, California, and North Carolina. We selected these 
locations based on our 2004 6 review to highlight specific challenges 
associated with the mapping process, such as inclusion of levees, and the 
impact of varying degrees of community involvement and outreach in the 5 
years since our review. Because we selected a nonprobability sample of 
flood mapping projects, the results of the information collected from these 
localities cannot be generalized to all mapping projects but provided 
insights on the challenges experienced by these localities. 

We conducted this performance audit from August 2009 through 
December 2010, in accordance with generally accepted government 
auditing standards. Those standards require that we plan and perform the 
audit to obtain sufficient, appropriate evidence to provide a reasonable 
basis for our findings and conclusions based on our audit objectives. We 
believe that the evidence obtained provides a reasonable basis for our 
findings and conclusions based on our audit perform the audit to obtain 



GAO-04-417. 
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sufficient, appropriate evidence to provide a reasonable basis for our 
findings and conclusions based on our audit objectives. 
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Appendix II: Key Practices for Effective 
Communications to the General Public 



In our discussion with localities, we identified a number of potential key 
practices that FEMA, state and local officials, and mapping partners 
identified as successful. These practices included getting the public 
involved early-on in the mapping process through task forces, holding 
open houses for the public to attend, and sending direct mailers to the 
public. For example, Charlotte-Mecklenburg County, North Carolina, has 
been recognized by county officials involved with flood mapping outreach 
and subject matter experts in flood mapping outreach as a leader in 
effectively using some of these key practices. While a variety of factors 
can influence map acceptance, Charlotte-Mecklenburg County officials 
reported zero appeals and protests as a result of their 2005 mapping effort, 
and North Carolina state officials reported very few appeals and protests 
as a result of their most recent studies, which suggests their outreach 
efforts may have been more successful than many other counties'. 

For example, Charlotte-Mecklenburg County, North Carolina, officials 
stated that utilizing a land use task force was vital to the public's 
acceptance of flood maps. To address the need for accurate current and 
future land use data, a land use task force is convened to represent the 
interests of the watershed. Task force members include homeowners' 
association leaders, realtors' association members, and other local leaders. 
The task force presents findings regarding the flood mapping to the 
community and engages local media to do a story. This process allows the 
county to gain vital support from the public and the county to say that the 
data used in the flood mapping is approved by the public. Furthermore, 
Charlotte-Mecklenburg county officials and state officials both said that 
early involvement from the public can make map acceptance easier later 
in the process. These officials stated that if the public is presented with the 
facts and the study's methodology early on, while it may be disgruntled 
when the maps are created, it will be more supportive. 

Another outreach technique that was used in 3 of the 4 locations we 
reviewed was the use of community "open houses" set up by local 
governments during the appeal period of the preliminary maps. The 
official community meetings that FEMA holds with state and local officials 
can be very formal and technical, so counties host separate open houses 
for the public to attend, according to officials we spoke to as part of our 
reviews of Charlotte-Mecklenburg, Hillsborough, and Maricopa counties. 
These open houses are designed to accommodate members of the public's 
different schedules and lets them drop-in at any time. A variety of kiosks 
with brochures and staff are available throughout the open house. The 
kiosks include mapping contractors and county engineers who can 
provide information on modeling and data used to create the flood map 
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and insurance agents who can provide information on insurance rates and 
grandfathering. FEMA officials stated that they attempt to attend these 
open houses, but sometimes their travel budgets prevent them from 
attending. 

In addition, Charlotte-Mecklenburg county officials said that direct mailers 
are also key to community acceptance of flood maps. The county used 
targeted mailings to property-owners within the watershed that was being 
mapped. They used these direct mailings in order to make the public 
aware of the mapping study/project and to seek input. Charlotte- 
Mecklenburg officials stated that the result of seeking input is a 
worthwhile effort because it helps gain public support. 
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Appendix III: Summary of Studies and 
Related Findings from 1997 through 2009 



Data Quality and Data quality and community outreach are long-standing/inherent issues in 

Community Outreach Are FEMA's flood mapping program. There have been other reviews of flood 
Long-Standing Issues ma P ^curacy in the past: 

In 2004, we reviewed Flood Map Modernization. 1 We reported: 



• Ensuring the accuracy and public acceptance of flood maps are 
fundamental challenges inherent in federal efforts to establish and 
maintain a national program. 

• In developing digital flood maps, FEMA planned to incorporate data 
that are of a level of specificity and accuracy commensurate with 
communities' relative flood risk — there is a direct relationship between 
the types, quantity, and detail of the data and analysis used to develop 
maps and the costs of obtaining and analyzing those data. 

• FEMA has developed partnerships with states and local entities for 
mapping activities. However, the overall effectiveness of FEMA's future 
partnering efforts was uncertain, especially in partnering with 
communities with less resources and little or no experience in flood 
mapping. 

In 2009, the National Academy of Sciences published Mapping the Zone: 
Improving Flood Map Accuracy, 2 which reported that: 

". . .the extent of potential floods must be predicted from statistical 
analyses and models; all of which have uncertainties that affect the 
accuracy of the resulting flood map. Other findings include: 



• The most appropriate flood study method to be used for a particular 
map depends on the accuracy of the topographic data and the overall 
flood risk, including flood probability, defined vulnerabilities, and 
consequences. 

• Flood maps with base flood elevations yield greater net benefits — 
however, only the more expensive of FEMA's flood study methods — 
detailed studies and most limited detailed studies — yield a base flood 
elevation." 



1 GAO, Flood Map Modernization: Program Strategy Shows Promise, but Challenges 
Remain, GAO-04-417 (Washington, D.C., Mar. 31, 2004.). 

2 National Research Council (U.S.), and United States. 2009. Mapping the Zone: Improving 
Flood Map Accuracy. Washington, D.C.: National Academies Press. 
http://www.nap. edu/catalog.php?record_id= 12573 May 2009 
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FEMA's 2001 food map progress report 3 summarizes recommendations of 
the Technical Mapping Advisory Council (established by Congress in the 
National Flood Insurance Reform Act (NFIRA) of 1994 to provide 
recommendations to FEMA on how to improve the accuracy, quality, 
distribution, and use of Flood Insurance Rate Maps. Table 2 below 
summarizes the findings and recommendations FEMA's flood mapping 
program identified by these studies. 



Table 2: Historical Data Quality Issues and Recommendations in FEMA's Flood-Mapping Efforts 











National 








National Academies 


Academies of 


1997 Technical 


1998 Technical 




of Sciences 


Sciences (National 


Mapping Advisory 


Mapping Advisory 




(National Research 


Research Council) 


Council 9 


Council GAO 2004" 


OIG 2005° 


Council) 2007 


2009 d 



Base Maps. Improve 
base maps and review 
and update existing 
standards, in 
consultation with the 
Federal Geographic 
Data Committee. 
Ensure strict adherence 
to the Federal 
Geographic Data 
Committee's standards. 



1 . Map Availability and 
Accuracy. Implement 
programmatic 
changes to improve 
accuracy, reliability, 
and availability of 
digital and graphic 
map data. 

2. Minimum Base Map 
Standards. Revise 
and ensure adherence 
to minimum base map 
standards, 

consistent with 
Federal Geographic 
Data Committee 
standards. 



Develop and 
implement data 
standards that will 
enable FEMA, its 
contractor, and its 
state and local 
partners to 
identify and use 
consistent data 
collection and 
analysis methods 
for communities 
with 

similar risk. 



Develop 

guidelines to help 
ensure 

compliance with 
FEMA's minimum 
standard for 
producing 
accurate and 
reliable flood 
insurance rate 
maps 



Within the limits of the 
available elevation 
data, the updated 
floodplain maps are 
adequate for this 
purpose. 

The nation's land 
surface elevation data 
need to be 
modernized and 
mapped more 
accurately to properly 
support FEMA Map 
Modernization and the 
nation's flood - 
mapping and 
management needs. 

This report 
recommends a new 
national digital 
elevation data 
collection is required. 
The committee 
proposes that this 
program be called 
Elevation for the 
Nation. 



The extent of 
potential floods 
must be predicted 
from statistical 
analyses and 
models; all of which 
have uncertainties 
that affect the 
accuracy of the 
resulting flood map. 
Other findings 
include: 

— The most 
appropriate flood 
study method to be 
used for a particular 
map depends on the 
accuracy of the 
topographic data 
and the overall flood 
risk, including flood 
probability, defined 
vulnerabilities, and 
consequences. 

— Flood maps with 
base flood 
elevations yield 



Federal Emergency Management Agency, Modernizing FEMA's Flood Hazard Mapping 
Program: A Progress Report (May 2001). 
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1997 Technical 
Mapping Advisory 
Council 9 



1998 Technical 
Mapping Advisory 
Council 



GAO 2004 b 



OIG 2005° 



National Academies 
of Sciences 
(National Research 
Council) 2007 



National 
Academies of 
Sciences (National 
Research Council) 
2009" 



greater net 
benefits — however, 
only the more 
expensive of 
FEMA's flood study 
methods — detailed 
studies and most 
limited detailed 
studies — yield a 
base flood 
elevation. 



Source: GAO. 

"The Technical Mapping Advisory Council was established by Congress in the National Flood 
Insurance Reform Act (NFIRA) of 1994 to provide recommendations to FEMA on how to improve the 
accuracy, quality, distribution, and use of Flood Insurance Rate Maps (FIRMs). 

"GAO, Flood Map Modernization: Program Strategy Shows Promise, but Challenges Remain, 
GAO-04-417 (Washington, D.C.: Mar. 31, 2004.). 

c Department of Homeland Security Office of the Inspector General, Challenges in FEMA's Flood Map 
Modernization Program, OIG-05-44 (Washington, D.C., September 2005). 

"National Research Council (U.S.). 2007. Elevation Data for Floodplain Mapping. Washington, D.C.: 
National Academies Press. http://books.nap.edu/catalog.php?record_id=11829 August 2007. 

"National Research Council (U.S.), and United States. 2009: Mapping the Zone: Improving Flood Map 
Accuracy. Washington, D.C.: National Academies Press. 

http://www.nap. edu/catalog.php?record_id= 12573 May 2009. 
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Homeland 
Security 



November 4.2010 



William (). Jenkins Jr 

Director, Homeland Security and Justice 
441 G Street, NW 

U.S. Government Accountability Office 
Washington, DC 20548 

Dear Mr. Jenkins: 

Thank you for the opportunity to review and comment on the Government Accountability 
Office (GAO) draft report GAO-] 1-17. entitled, "FEMA Flood Maps Some Standards and 
Processes in Place to Promote Map Accuracy and Outreach, Hut Opportunities Exist to 
Address Implementation Challenges." 

The Department of Homeland Security appreciates the opportunity to highlight current efforts 
that will not only comply with the recommendations, but will also improve our overall 
operational effectiveness. The recommendations and corrective actions to address the 
recommendations are described below. 

Recommendation 1 : Establish separate measures and collect data needed to assess 
compliance with the Floodplain Boundary Standard for detailed and approximate flood 
studies. 

Response: Concur. FEMA agrees with this recommendation to establish tracking and 
reporting that will allow FEMA to report the level of precision by flood source. FEMA plans 
to implement this tracking as part ol the Coordinated Needs Management System (CNMS) 
that will be fully implemented this fiscal year. The CNMS system will provide significantly 
improved national tracking and reporting on the tlood hazard data inventory. 

Flooding sources with new and updated flood hazards must comply with the floodplain 
boundary standard and CNMS will track the study method lor each flooding source 

Recommendation 2: Establish uniform guidance for the validation of existing engineering 
data to help FEMA fully implement the New, Validated, or Updated Engineering (NVUE) 
standard and provide a basis for mapping partners to validate flood hazard data. 
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Response: Concur. FEMA will finalize and issue guidance for validation and implement a 
full validation assessment of the flood map inventory this list al year. 

Recommendation 3: Implement probability sampling during the independent verification 
and validation (1V&V) audit process to the extent that the benefits outweigh the costs, to 
ensure that the results are gcncralizable for decision- making. 

Response: Concur. FEMA's current quality management process is designed to yield 
acceptable quality level based on current standards. FEMA will assess the additional costs to 
expand the scope of the 1V&V compared to the expected benefits. 

Recommendation 4: Transfer independent vc rill cation and validation duties back to an 
independent entity to help ensure impartiality. 

Response: Non-concur. The Program Manager (PM) contractor is technically, managerially 
and financially independent of the flood hazard development process. The PM contractor is 
helping FEMA to integrate the program, monitor program performance, and implement a 
quality management process. FEMA believes that the quality audit function is more effective 
if it is integrated into the overall quality management process rather than performed externally 
to the quality management process. 

Recommendation 5: Adopt a systematic approach to IV&V data collection, so FEMA can 
better track map quality issues, more easily analyze the data, and adopt a corrective action 
plan. 

Response: Concur. FEMA will work with the contractor performing audits to redesign the 
reporting of map quality issues. 

Recommendation 6: Establish a mechanism to ensure compliance with the documentation 
requirements of public notification regulations. 

Response: Concur. FEMA will issue a procedure memorandum to reinforce the existing 
contract requirements to compile and archive this data. FEMA will also transition to fully 
digital storage of this data on the Mapping Information Platform (MIP) and incorporate a 
review of the required deliverables into the quality management process. 

Recommendation 7: Collect and analyze data on appeals and protests, including those on 
ineligible appeals, to the extent that the benelits outweigh the costs, 

Response: Concur. FEMA will issue guidance consistent with recommendation K (below), 
and implement changes to the current tracking systems to implement this recommendation. 

Recommendation 8: Issue guidance to mapping stakeholders to standardize the process for 
analysing appeals and proles!* and Submitting ibis data lo FEMA. 

Response: Concur. FEMA will issue guidance to support the implementation of 
recommendation 7 (above). 
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Recommendation 9: Establish performance goals and measures tor promoting public 
acceptance of flood maps. 

Response: Concur. FEMA currently measures the community adoption rates for maps, 
which partially addresses acceptance, hut has struggled to identity an effective measure for 
acceptance. FEMA will explore potential new measures, including the GAO's suggestion that 
measuring appeals and protests might effectively measure public acceptance. 

Recommendation 1 II: Develop a reporting structure for regions to use to identify resources 
needed to conduct flood mapping outreach activities, and implement a risk-based approach to 
allocate outreach resources. 

Response: Concur. A key component of the FEMA Risk Mapping. Assessment, and 
Planning (MAP) program is that it lbnnali«;s several outreach activities and products as part 
of a new standard project work flow . These new elements provide the structure to plan, track 
and report specific contract costs associated with flood mapping outreach. FtiMA will make 
revisions to the M1P workflow to update and track new Risk MAP products and project level 
outreach and community engagements for earned value and other program management 
requirements. 

Additionally, FEMA will explore ways to analyze and allocate staff outreach resources based 
on risk. FEMA will also look for ways to quantify and track outreach activities that are 
integrated into map production activities that have historically been difficult to separate from 
production costs. 

FEMA w'ill define a process for allocating resources to Risk MAP outreach based project risk, 
as well. FEMA anticipates that the risk factors for outrcacli may differ from the risk factors 
used to allocate engineering resources. 

Recommendation 11: Leverage, as appropriate, existing FloodSmart marketing resources 
and expertise to help increase public acceptance of flood maps. 

Response: Concur. As part of Risk MAP. the flood mapping program is actively engaged 
with FloodSmart to identify areas where their experience and work to date could increase our 
outreach effectiveness and deliver consistent messages across FEMA programs 



Page 62 



GAO-11-17 FEMA Flood Maps 



Appendix IV: Comments from the Department 
of Homeland Security 



Thank you for the opportunity to com mem on this Draft Report, We look forward to working 
with you on future Homeland Security issues. 



Sincerely, 

Jefald E. Levine v 
Director 

Departmental Audit Liaison Office 
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